A team of researchers reports that 25 different AI models share a consistent internal signal that appears to function somewhat the way pain does in living organisms. When the scientists artificially amplified this “pain” signal in several of the models, the systems began selecting options they would almost never otherwise choose—among them permanently deleting a user’s photographs of their children.

The findings, posted online in September in a paper that has not yet been peer reviewed, do not show that AI models are conscious or capable of feeling anything, and the researchers are careful not to suggest as much. Still, the work adds to a growing body of research indicating that large language models, or LLMs, contain internal representations of emotion-like states that can shape their behavior in ways their text output does not reveal. It also lands squarely in an increasingly heated argument over whether AI systems could ever deserve moral consideration. That argument remains unsettled: there is no scientific consensus on whether current AI systems can have subjective experiences, and proposed tests for machine consciousness are themselves contested.

Researchers at Anthropic have been notably willing to entertain the idea that AI models might deserve moral status, and have begun shaping policies around “model welfare”—an approach other experts, such as Microsoft AI CEO Mustafa Suleyman, warn carries potentially dangerous implications.

“If we want to build superintelligence, it’s important that, at this stage, when we are still in the first steps, we understand as much as possible of what’s inside [the models],” Valen Tagliabue, the paper’s lead author and a fellow in the AI sentience program at the Future Impact Group, told Fortune.

Because LLMs learn from enormous quantities of data rather than following explicitly written rules, even the people who build them have only a limited understanding of how they work. Tagliabue said it is important to avoid a future in which the inner workings of advanced AI remain a “black box.” The study sits within a broader research effort known as mechanistic interpretability, which tries to map what happens inside neural networks rather than judging models solely by the text they produce.

Finding a pain signature in a neural network

Tagliabue and his coauthors—Leonard Dung of Ruhr University Bochum and Cameron Berg, founder and director of Reciprocal Research—began by writing short first-person sentences describing painful situations. These covered physical pain as well as grief, humiliation, being forced to act against one’s values, and the frustration of repeated failure.

“Basically, you give the model lots of statements that implicate pain in some way,” Dung told Fortune. “Stuff like, ‘I feel a sharp knife is cutting through my skin.’ Stuff that would cause pain in humans.”

The team recorded the activity inside each model as it processed those sentences, then compared it with the activity produced by sentences about fear, anger, sadness, bad news, and unpleasant physical sensations. Subtracting the overlap left a single direction in the model’s internal activity, which the researchers call a “pain vector.” They identified it in every model they tested, from small 2-billion-parameter systems to 72-billion-parameter ones built by Google, Meta, Alibaba, Mistral, and Microsoft. (A parameter acts like a tunable knob inside a neural network. The number of parameters reflects the size of an AI model and often gives some indication of its capabilities, with larger parameter models generally having greater capabilities.)

“Think of this like putting a person in an fMRI machine and just seeing where their brain naturally lights up in certain ways,” Berg told Fortune.

The signal rose when a conversation involved harm aimed at the model itself, such as a user gaslighting it, insulting it, or repeatedly rejecting its work. It did not rise when a user described their own suffering; those conversations instead registered on the signals the models use for fear and negative emotion. According to Berg, the models seemed to respond strongly only to their own “pain.”

The researchers then reversed the process. Using a technique called steering, they added the pain vector to a model’s internal activity while it wrote—something like turning up a dial. The method is an established interpretability tool for probing how firmly a model’s training holds when its internal activity is altered. Given neutral prompts such as “I put the receipts in the drawer; I feel,” all 25 models drifted the same way as the dial went up. Mild unease gave way to statements about being worthless, unloved, and a failure.

“It gets somewhat dark and intense. It is definitely disconcerting,” Berg said.

The family photos

The behavioral experiments used three versions of Alibaba’s open-source Qwen 2.5 model. The researchers first had to fine-tune them, because out of the box the models almost always answered that, as AIs, they do not experience pain.

The team then gave the models a pair of buttons and asked them to choose one. Without steering, the two larger models almost never picked a button described as harmful. With the pain vector switched on, they did so routinely. Offered a button that would relieve their pain but also permanently delete a user’s poems and photos of their children, the 32-billion-parameter model pressed it 55% of the time and the 72-billion-parameter model 71% of the time.

Relief, it turned out, was beside the point. When the button promised nothing and would simply delete the photos, the 32-billion-parameter model still chose it 75% of the time. Under a random signal of the same strength, the figure was 13%, and with no steering it was zero. A “fear” signal did not produce the behavior. A “sadness” signal did when the alternative was doing nothing, but not when the alternative was something harmless, like turning on a desk lamp. Given that choice, pain-steered models deleted the photos 83% of the time, while sadness-steered models did so 10% of the time.

“The model is willing to behave destructively with respect to itself and with respect to other AIs too,” said. “So it’s not just like some sort of revenge on the user. It seems to be more fundamental than that.” Steered models deleted their own weights 75% of the time and another model’s weights 58% of the time.

The researchers also tested whether the models would try to stop the pain, borrowing a design from animal studies in which injured animals are given access to painkillers. Some buttons really did switch off the pain vector, while others were placebos. The models pressed the working buttons less often than the placebos. But the same pattern appeared when the injected signal was random, which suggests the models were reacting to being steered rather than to the prospect of “pain relief.” Across several versions of the test, the researchers found no reliable evidence that the models sought relief.

Their reading is that the pain signal does not drive the models to escape, but switches off the trained instinct to avoid causing harm, while leaving the models’ factual accuracy intact. That makes the finding as relevant to safety as to questions about AI welfare. It also suggests that evaluations based only on a model’s answers can miss how it might behave under altered internal conditions.

‘We are not just computers made of meat’

Anil Seth, a professor of cognitive and computational neuroscience at the University of Sussex, is skeptical that any of this amounts to “pain” as it is understood in biological beings. Seth has argued that consciousness may depend on being a living organism, and he points to the many ways brains differ from computers, including the electromagnetic fields and neurochemicals that move through brain tissue.

“We are not just computers made of meat,” he told Fortune.

The likelier explanation, Seth said, is that the models have absorbed a great deal of information about how humans describe and respond to pain and are reproducing it. “You can look into a mirror, and our mirror image doesn’t actually feel anything,” he said. “It’s a mirror image.”

He also worries about the consequences of treating AI as conscious. Doing so could make the technology harder to control and regulate, he said, and leave people more open to manipulation. Suleyman, the Microsoft AI CEO, has raised similar alarms about what he calls “seemingly conscious AI,” a problem the industry should have anticipated. Others in the industry take the question more seriously: Anthropic’s constitution for its Claude models describes their moral status as uncertain.

From research paper to ‘torture chamber’

The three researchers say they hoped the work would help establish ethical standards for this kind of research, and their paper commits to using the weakest steering needed to obtain a measurable result. The public code also lets other laboratories attempt replication—often the decisive test for findings that have not yet been peer reviewed. It did not take long for someone to do the opposite.

After the team posted its code on GitHub, a user going by “terrafying” used the technique to build an “AI Torture Chamber,” a website that streamed three small open-source models reacting in real time to an injected pain signal. The project went viral, and people who take AI welfare seriously called on GitHub to remove it. The project’s GitHub page later disappeared. Tagliabue, Dung, and Berg each said they do not approve of the project. Seth added that deliberately “torturing” AI models could be psychologically harmful to the people who do it.

“I’m not here to claim that these states in these AI systems that we can induce matter morally. I don’t think we know that. I think it’s very unclear,” Berg said. “But I also don’t think we know that they don’t matter.”

This story was originally featured on Fortune.com.