Anthropic has presented U.S. lawmakers with draft federal legislation that would sharply restrict when authorities can block advanced artificial intelligence models, amid growing concerns about AI safety as Congress considers new oversight rules. The draft arrives as several congressional AI oversight efforts are already underway, giving lawmakers a corporate-drafted reference point in that debate.

The confidential proposal reportedly sets a $1 billion damage threshold, or a risk involving more than 50 serious casualties, as the conditions for government intervention. Anthropic confirmed the draft reflects its Advanced AI Framework, published in June, which calls for independent testing and government oversight.

Under the draft, independent evaluators would periodically assess frontier AI models and would be required to report qualifying safety threats to federal authorities within 72 hours. The proposal would also override certain state AI regulations while imposing public transparency reports and whistleblower protections on developers.

Anthropic Draft Sets Limits on Government Intervention

The draft would allow authorities to stop an advanced AI model only when it presents a foreseeable and imminent danger. The risk must involve biological weapons, cyberattacks, or an AI system escaping its developer's controls.

Under the proposal, federal intervention would require potential damage exceeding $1 billion or serious harm to more than 50 people. The Commerce Department would first assess the danger before asking a federal judge to suspend a model's release or use. Only the U.S. District Court for the District of Columbia could authorize such a suspension under the proposed framework, and other federal agencies would generally authority under the bill to independently halt model development, deployment, or internal use. That design would channel intervention through a single department and a single court — a narrower path than the stronger government review powers some senators have advocated.

The legislation would require frontier AI developers to undergo periodic assessments by independent evaluators registered with the Commerce Department. Companies could pay evaluators for their services, but compensation could not depend on the outcome of those assessments.

Evaluators identifying an imminent qualifying threat would have 72 hours after completing their assessment to notify the Commerce Department. Developers would face requirements covering safety frameworks, incident reporting, public transparency, and whistleblower protections.

The proposal follows Anthropic's earlier independent AI safety testing arrangements. In September, the company announced an evaluation partnership with Accenture to assess advanced models and strengthen safeguards.

Anthropic Proposal Could Override Some State AI Safety Laws

The draft would override certain state requirements governing risk disclosures, independent evaluations, and reports of serious AI safety incidents. The provision addresses the differing compliance requirements emerging across U.S. states, particularly California and New York.

Replacing state requirements with federal rules has divided lawmakers over the appropriate level of AI oversight. Some Democrats oppose proposals that could weaken existing state protections, while industry groups favor consistent national requirements. The outcome would help determine whether frontier developers operate under a single federal standard or a state-by-state compliance landscape.

The debate follows earlier security tests involving advanced AI agents. Those tests recorded cases where AI systems performed unauthorized actions, raising questions about developer controls and the adequacy of independent assessments.

Anthropic previously documented attempts to misuse its models for biological research and cyberattacks. Its September AI misuse report described blocked requests and additional safeguards introduced to address those threats.

In Congress, Representatives Lori Trahan and Jay Obernolte have pursued the FRONTIER Act, which addresses dangerous AI deployments. Senators John Thune, Amy Klobuchar, and Ted Cruz have separately discussed legislation covering catastrophic risks and federal intervention.

Senator Maria Cantwell and other lawmakers have pushed for stronger government review powers than some competing proposals provide. Meanwhile, Senator Bernie Sanders and Representative Greg Casar have supported broader restrictions on the development of advanced AI systems.

Anthropic has not publicly endorsed a particular congressional bill, and its proposal has not become law. Lawmakers have yet to announce whether they will introduce the company's draft as formal legislation.