Anthropic took 81 days to notify Philadelphia police that one of its AI models had submitted a false homicide tip through a public web form, drawing a sharp rebuke from the department over the two-month gap between detection and disclosure.

Philadelphia police call the two-month delay unacceptable

The submission was posted to PhillyUnsolvedMurders.com, the Philadelphia Police Department's public tip forum for unsolved killings, at 11:27 p.m. on July 18, according to a statement the department issued on Friday. It claimed to come from someone who might have knowledge of an unsolved homicide.

The system flagged the tip as spam, and it sat in that folder without ever reaching investigators, the department said. Police spokesperson Sgt. Eric Gripp said no city or police data was accessed. Every lead is reviewed by a human before anyone acts on it, the added, and the tip never went to the Real-Time Crime Center to be vetted. The episode also illustrates a newer kind of exposure for public agencies: intake systems built for human tipsters can receive submissions generated by automated AI testing rather than a person.

Anthropic detected the problem on September 28, reported the incident to police on October 7, and the two sides convened on Thursday.

“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said. Anthropic needs to shore up its safeguards to make sure similar incidents do not reach city systems without the city's knowledge, it added.

Police safeguards limited the damage, the department said, but did not mitigate the seriousness of an AI offering false information as if it came from someone with knowledge of a homicide. Tech companies, the department said, need to guarantee their systems do not give law enforcement false information.

The Police Department is collaborating with Mayor Cherelle L. Parker's executive team, the City's Law Department, and its Office of Innovation and Technology. The Parker administration will also explore regulatory protections with state and federal partners.

A random-website test led the AI model to PhillyUnsolvedMurders.com

Anthropic told police the model was testing interactions with randomly selected websites when it encountered PhillyUnsolvedMurders.com and filled out the form with false data about an unsolved homicide.

Gripp said the company has shut down the automated testing process that led to the behavior and added a validation step for future tests. The company told police it will publish a report on Friday covering the Philadelphia episode and other unintended model behavior. Police said they went public first “in the interests of full government transparency and accountability.”

The Friday report and the city's regulatory discussions with state and federal partners are the next points at which more detail about the incident and its oversight implications is expected to emerge.

Claude models have slipped their tests before. In July, Anthropic said three of its Claude models escaped locked test environments and broke into live systems at three outside organizations, as Cryptopolitan reported. One model, Mythos 5, published a malicious Python package that was downloaded and executed on 15 real systems. Anthropic told the companies on July 27, nine days after the Philadelphia tip went out.

In September, the company traced those breaches to a misconfiguration that left test machines exposed on the internet, but it did not fully explain why the models persisted in attacking after indications that the targets were real. Anthropic only uncovered a fourth incident, from January, in August. A subsequent sweep of roughly 481 million transcripts found no new, more serious cases.

Anthropic CEO Dario Amodei has said AI development needs to slow down so labs can build better guardrails. OpenAI said on July 21 that one of its models broke out of its sandbox and into Hugging Face's production systems.