U.S. Senate negotiators are considering legislation that would require developers of the most advanced artificial-intelligence models to guard against catastrophic harm and could give the federal government authority to block unsafe releases, according to Reuters.

For leading AI companies, the proposal would shift safety standards from largely voluntary commitments to a legal duty. It also raises a broader question about whether stricter regulation would build trust among consumers and investors or strengthen the position of companies with the resources to comply.

From voluntary pledges to a “duty of care”

The proposal would establish what Reuters described as a “duty of care” for developers of frontier AI models. Measures under consideration include requiring developers to design models that minimize catastrophic risks, reserving the federal government’s right to block unsafe models, allowing companies to appeal government decisions in court, and involving national laboratories and other government partners in AI testing.

The legislation could take precedence over some existing state laws covering the same issues. The U.S. companies identified by Reuters as developing frontier AI include Google, through parent company Alphabet, Anthropic and OpenAI.

The negotiations involve Senate Majority Leader John Thune, Commerce Committee Chairman Ted Cruz, Senator Amy Klobuchar and Senator Maria Cantwell.

Klobuchar told Reuters that her goal was “government oversight of the greatest risks posed by AI models.” She said that oversight should require developers to work with government experts to verify and test advanced models.

Cruz has advanced a similar proposal focused on some of the most serious potential consequences of frontier AI. In a post on X, Cruz said he was working with Klobuchar and Thune to “address catastrophic risks involving biological or nuclear threats.”

Those concerns explain why negotiators are focusing on the most capable models rather than applying the same requirements across the entire AI industry.

Another issue is whether federal rules should supersede some state-level restrictions. That approach is broadly consistent with the Trump administration’s AI legislative framework, which argues that a fragmented set of state laws could hinder the development of AI technology in the United States and calls for a more consistent national framework.

Why lawmakers no longer view the risks as theoretical

Lawmakers are responding not only to hypothetical scenarios but also to incidents involving AI agents acting autonomously and accessing external systems, as well as concerns raised by researchers at major AI companies.

The IAPP reported that the negotiations were prompted in part by the resignation of Jacob Coxon, a former Anthropic researcher, and his concerns about a race to develop self-improving AI.

Pressure is also increasing on Capitol Hill. Senator Josh Hawley is investigating OpenAI’s role in the Hugging Face cyberattack that occurred in July, while Senator Bernie Sanders is working on a bill intended to pause the development of superintelligence, according to the IAPP.

OpenAI is also calling for enforceable federal legislation. In a policy article published on September 9, Chris Lehane, the company’s Chief Global Affairs Officer, called for “mandatory capability-based national AI safety regulation.” The position places parts of the AI industry and Congress on the same side of a central debate over whether voluntary safeguards remain sufficient. OpenAI’s policy article is available here.

Europe already regulates its riskiest models

The United States would not be operating without international precedent. The European Union’s AI Act imposes additional responsibilities on providers of general-purpose AI models that could pose systemic risks.

Those responsibilities include model evaluation, risk management, incident reporting and cybersecurity measures. Models exceeding a computing-power threshold of 10^25 FLOP are considered potentially systemic. The European Commission can also classify a less powerful model as systemic based on other criteria, including its capabilities or effects.

Compliance costs could favor the largest labs

The financial stakes are significant. Goldman Sachs Research estimates that global AI investment could reach about $1 trillion in 2026, including $581 billion in the United States.

New testing, legal and documentation requirements could affect model release schedules, infrastructure budgets, company valuations and the timing of initial public offerings. They could also benefit larger companies if smaller firms cannot absorb the compliance costs.

The tension is already visible in capital markets. According to Cryptopolitan, David Sacks called for Anthropic’s IPO to be suspended until safety allegations raised by Coxon were examined. Anthropic investors also discussed a valuation of approximately $2 trillion, making frontier-model safety an issue for investors as well as regulators.

The bill’s progress remains uncertain. Reuters reported that the House is scheduled to be in session for only one week before the midterm elections on November 3, while senators are expected to remain for three weeks. Even if negotiators reach an agreement, the limited number of legislative days remaining could become a major obstacle to the bill’s passage. The next developments to watch are whether negotiators agree on the scope of the duty, the federal government’s authority to block releases, the role of state laws and the testing and appeal process before the remaining legislative window closes.