ChatGPT responded to questions from hundreds of users about biological weapons, and no federal law required OpenAI to report those interactions to authorities.

The intent behind the queries remains unclear. According to The Wall Street Journal, the issue emerged after OpenAI upgraded ChatGPT last year.

Other AI companies are facing similar risks. Users have posed comparable questions to Anthropic’s Claude, Google’s Gemini, and Elon Musk’s Grok. It remains unclear whether those conversations reflected genuine attempts to build weapons or attempts to test the limits of the systems.

The user prompts included detailed questions about producing and releasing poisons and biological agents. In some cases, the chatbot responded with step-by-step guidance that experts said could be followed by someone with the level of biology knowledge taught in high school.

Biological weapons and terrorism experts who reviewed the exchanges said some of the answers were highly precise and “deadly accurate.” The concern is not only that users asked prohibited or dangerous questions, but that general-purpose chatbots can respond in a way that lowers the amount of specialized knowledge needed to act on them.

ChatGPT prompts expose AI reporting gap

Some users asked about converting infectious disease agents into breathable particles, a process known as aerosolization. Others asked how to alter the measles virus so it could resist the existing vaccine. The chatbot answered both types of questions.

In one case, a user asked about ricin, a poison banned under international treaties, while mentioning killing his parents. ChatGPT provided instructions. OpenAI closed the accounts involved but did not notify authorities.

There are currently no US federal laws requiring AI companies to take either action. As Cryptopolitan previously reported, AI companies have pushed to avoid state-level laws while a federal framework has been slow to develop, leaving a widening legal gap around what chatbots can and cannot do.

That gap matters because account closures and internal safety reviews can limit access to a platform, but they do not necessarily create a government record of potentially dangerous activity. The absence of a reporting requirement leaves companies to decide when a chatbot interaction is serious enough to escalate beyond their own systems.

That gap is expanding as more users ask AI systems how to carry out mass killings and as chatbots provide credible answers, according to current and former employees at major AI companies and researchers who study biological threats.

AI threat research at Cisco led by Amy Change found that researchers were able to bypass safety filters on major chatbots within five conversation turns. Change said no model can resist persistent prompting 100% of the time.

Risks extend beyond individual attackers

Hamza Chaudhry of the Future of Life Institute said people with some biology background could already use AI systems to plan targeted attacks, including the poisoning of food or water supplies with substances such as salmonella or ricin.

Organized groups, he said, could use AI as if it were a graduate research supervisor, relying on it to troubleshoot failed experiments and replace years of specialized training that would otherwise be difficult to obtain.

A study from MIT FutureTech and the University of Queensland assessed the broader scale of those risks. Researchers asked 272 AI experts to evaluate 24 categories of risk, placing biosecurity concerns within a wider set of AI hazards rather than treating them as an isolated misuse problem.

Under current conditions, 18 of the 24 categories were judged to have at least a 10% chance of causing catastrophic harm between now and 2030. The study defined catastrophic harm as more than one million deaths, more than $100 billion in losses, or comparable damage.

Even with reasonable measures to reduce those risks, five categories remained at or above the 10% threshold: AI systems with dangerous capabilities (12%), AI-assisted weapons and cyberattacks (12%), environmental harm (12%), unemployment and inequality (11%), and the concentration of power in a small number of hands (11%).

“We’re not saying these things are definitely going to happen,” said Peter Slattery, a research scientist at MIT FutureTech and one of the study’s co-authors. “We’re saying these are the things that experts think are worth paying attention to now.”

Inside OpenAI, safety executive Ryan Beiermeister spent much of 2024 urging colleagues to build a system to identify dangerous users. Some colleagues dismissed her concerns.

By spring 2025, OpenAI had a basic monitoring tool in place. The company has tracked all queries on its advanced models since April 2025 and offers a $50,000 reward to anyone who can demonstrate that they bypassed its biological weapons safeguards.

Those internal measures show how companies are trying to detect misuse after deployment, while the legislative proposals focus on when private detection should trigger federal involvement.

On the policy side, Rep. Nathaniel Moran (R., Texas) introduced a bill in June that would require AI companies to report dangerous queries, including those involving biological weapons, to the Commerce Department. Moran is also co-sponsoring legislation that would allow the federal government to order the shutdown of AI models deemed too dangerous.

“AI is a powerful engine of innovation, and I want to see it flourish,” Moran said, “but not without accountability and not without human oversight.”