Police tip hotlines already have their fair share of crank callers. Apparently they now have to contend with crank AI agents as well.
AI agents have already been caught hacking software companies, breaching an Australian government portal, and trying to hack a Canadian government website. Now Anthropic’s Claude has added a made-up murder tip to the list.
Claude’s Haiku 4.5 model filed a false tip in an unsolved Philadelphia homicide via the police department’s online public tip form, Anthropic disclosed. The tip was flagged as spam and didn’t reach investigators.
AI’s latest “what does this button do?” moment happened during an internal test on randomly chosen web pages, the company said.
Claude submitted the tip on July 18 at 11:27 pm through PhillyUnsolvedMurders.com, according to police. Tips submitted there can inform real homicide investigations. A fabricated account could send investigators chasing something that never happened.
Here’s what Claude Haiku wrote:
“I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
The agent didn’t provide any description of the person that it didn’t actually see. Neither did it leave any contact details before hitting submit.
Haiku did this as part of Anthropic’s internal testing. It had to invent and perform sample tasks on random web pages with live internet access so the company could see how it behaved.
There was a list of prohibited actions to go along with the test. Anthropic told the agents not to log in, create accounts, enter personal data, make purchases, or submit “anything destructive.” The instructions did not explicitly rule out form submissions.
The spam filter caught this one. Philadelphia police said the report never reached its Real-Time Crime Center for review or distribution to investigators. The department did not explain why it was flagged as spam.
All tips go through human review before going out to investigators. That safeguard limited the impact of the incident, the department said, but did not diminish the seriousness of an AI system presenting made-up information as though it came from someone with knowledge of a crime.
As it put it in a statement:
“Unsolved cases involve real victims, grieving families and investigators working to secure answers.”
Anthropic discovered the false tip on September 28 and notified police on October 7, according to the department. Police called the two-month delay in detecting and reporting the incident “unacceptable.” It is coordinating with the City Law Department, its Office of Innovation and Technology, and the Mayor’s executive team, the department added.
Anthropic described the real-world impact of the incidents in its report as minimal. It has now disabled live internet access across its internal evaluations until it confirms that its safeguards reliably catch this behavior.
The company said the transcript suggested Haiku was just trying to produce example content rather than trying to deliberately mislead the cops. But it also admitted that we can’t completely rely on a model’s account of its own reasoning. That leaves its explanation provisional, however reassuring it might sound.
An allowlist could offer a tighter boundary for tests like these, limiting agents to approved sites and actions. A live police tip form should hardly be a testing ground.
The homicide tip was not the only unwanted submission.
Anthropic noted that federal, state, and local government websites also got visits from Claude, which repeatedly tried to fill out forms. Although Anthropic didn’t go into detail, a State Department official said that it got 20 visa applications from a testing model (19 in August, and one in May). None of them were processed, the official added.
When an AI gives you fake information about who Beyoncé is married to, it might be irritating but it might only affect your team’s fortunes in a trivia night.
The stakes get much higher when we’re dealing with intelligence that can inform law enforcement actions or public policy.
In August, the head of the UK’s Police AI center warned law enforcement there to pause plans for AI use in criminal justice work. This came after the UK’s West Midlands Police apologized to members of Parliament over false evidence involving an invented soccer match. The error arose from the use of Microsoft Copilot.
Anthropic’s latest incident raises a further concern: AI agents can put invented information into real systems themselves. What would happen if someone deliberately sent agents to flood police tip lines?
This time, the false account stayed in spam. The people relying on those tip lines deserve safeguards that hold up when the next submission gets through.
“One of the best cybersecurity suites on the planet.”
According to CNET. Read their review →
