OpenAI told staff this week that it would consider slowing work in the most advanced areas of artificial intelligence. Chief Executive Sam Altman extended the proposal to rival laboratories, including Anthropic, which has separately acknowledged growing AI risks after reporting that researchers linked to state institutions attempted to use its Claude models for biological-weapons research.

OpenAI said it has already paused some internal training runs and slowed parts of model development because of safety concerns. The successive disclosures have intensified debate over whether the AI industry is advancing too quickly, particularly as both companies continue to signal possible public offerings.

Why does Altman want to slow AI development?

At a company-wide meeting, Altman told OpenAI employees that the company and other willing laboratories could slow the release of frontier models. According to sources cited by Bloomberg, he acknowledged that the arrangement would be voluntary and that not every company would agree to participate. That voluntary approach contrasts with OpenAI’s later call for mandatory, capability-based rules, making coordination among competing laboratories and the scope of any government requirements central issues in the debate.

There is precedent for skepticism about voluntary pauses. In March 2023, Elon Musk, Steve Wozniak and more than 1,000 other people signed an open letter calling for a six-month halt to advanced AI work. Media reports at the time said the participating laboratories did not ultimately implement the moratorium they had jointly proposed. The letter is available through Axios.

OpenAI Chief Scientist Jakub Pachocki made a similar appeal in a September 6 post, asking AI companies to coordinate “to slow down future development as needed.” Pachocki said he hoped voluntary slowdowns would “become commonplace until shared safety bars are established.”

Altman said in July that he had told White House officials about the need to throttle AI development. He and Anthropic Chief Executive Dario Amodei had also previously signed a one-line statement declaring AI to be a societal-scale extinction risk. The statement is published at AIstatement.com.

Could AI models support biological-weapons research?

AI models have advanced to a point where they may be capable of supporting biological-weapons research. Earlier this week, OpenAI said its models had solved the Navier-Stokes mathematics problem, although the claim was made under contested terms.

Anthropic added to the debate in a 154-page threat-intelligence report released on September 10. The report described five cases in which researchers attempted to use Claude’s capabilities to support biological-weapons work. Anthropic said the attempts came from “unsupported regions” and involved efforts to bypass location restrictions and conceal the purpose of the activity. The company said it removed the accounts but did not identify the countries or institutions involved. The report’s section on biological misuse is available on Anthropic’s website.

Growing concern among AI researchers

The reports came amid a series of public warnings from people inside and formerly associated with major AI laboratories. On September 8, researcher Jacob Coxon resigned and accused OpenAI and Anthropic of “gambling with our lives.” He said some colleagues believed the technology could “kill us all by the end of the decade” and warned of human extinction by 2030. His post received more than 150 million views.

On September 10, former employees of Anthropic and Google’s DeepMind expressed similar concerns and called for greater transparency. A petition launched in late July to establish a mechanism for slowing AI development received signatures from more than 1,000 workers across major AI companies.

OpenAI is now urging Congress to adopt mandatory, capability-based safety rules covering testing, independent evaluations, cybersecurity and incident reporting. “The prospect of AI-accelerated AI development demands more than voluntary commitments,” Chief Global Affairs Officer Chris Lehane wrote. The proposals put the focus on how advanced systems would be tested and monitored, rather than only on whether companies agree to pause development.

California acted first. Governor Gavin Newsom signed SB 813 and AB 1405 on Wednesday, while companion legislation, AB 1864, seeks to protect against AI-enabled biological threats.

In Washington, Senator Bernie Sanders introduced legislation that would pause advanced AI development and ban artificial superintelligence. He told the BBC that disregarding scientists’ warnings would make someone “a moron.”

President Donald Trump opposed that approach, saying on Thursday that he was concerned “if we don’t win AI, we’re going to be put in a very bad position.”

Originally published by Cryptopolitan.