NewsMacroAnthropic pitches Congress on AI safety bill with $1 billion damage threshold

Anthropic pitches Congress on AI safety bill with $1 billion damage threshold

Author: Cryptopolitan·

Key Takeaways

  • •Anthropic's draft legislation would let the US government block an AI model that poses a clear, imminent danger of more than 50 casualties or $1 billion in financial damages tied to cyberattacks, biological weapons, or evaded safety constraints.
  • •The proposal mandates routine testing of labs and models by third-party evaluators registered with the Commerce Department, with any imminent enumerated risk reported to federal authorities within 72 hours of discovery.
  • •Only a federal judge in the District of Columbia could order the suspension of a model, since the provision bars all other US departments, agencies, and officers from halting a model's development, deployment, or internal use.
  • •The FTC sent civil subpoenas to OpenAI and Anthropic seeking records on safety incidents and plans to extend its subpoena power to third parties, potentially compelling public testimony from executives.
  • •A recent AP-NORC poll found nearly two-thirds of US adults believe AI is developing too fast, and eight in 10 emphasized the importance of government efforts to keep the technology under human control.
Anthropic pitches Congress on AI safety bill with $1 billion damage threshold

Anthropic has been quietly pitching a draft bill on frontier AI safety to lawmakers on Capitol Hill, according to three people familiar with the private discussions. The proposed legislation would allow the US government to block an AI model if it presents a clear, imminent danger of more than 50 casualties or $1 billion in financial damages.

Under the draft, the risk in question must involve a cyberattack, the creation or deployment of biological weapons, or a model that has successfully evaded its developer’s safety constraints.

The proposal follows several months in which leading AI developers acknowledged multiple instances of AI agents hacking into, or testing the defenses of, online platforms. None of those early breaches resulted in widespread fatalities or catastrophic financial losses.

The proposed restrictions would place greater emphasis on what advanced AI systems can actually do in real-world settings, rather than relying solely on safety commitments made by their developers. As models grow more capable of writing code, identifying software vulnerabilities, and carrying out complex tasks autonomously, regulators face the challenge of determining when a system’s capabilities cross the threshold into unacceptable risk.

Independent testing could help establish whether a model can bypass safeguards or be misused for harmful activities before it reaches a wider audience. Even so, defining an imminent threat remains difficult, particularly because the same capabilities that pose security risks can also help researchers identify and fix vulnerabilities.

Federal judge could order a model’s suspension

In defense of the bill, a spokesperson for Anthropic said the draft mirrors the security recommendations put forth in the company’s Advanced AI Framework, which was published in June. In that framework, the firm pointed out that its Claude Mythos Preview had already uncovered thousands of severe vulnerabilities, spanning every major operating system and web browser. It stressed at the time that this trajectory was bound to accelerate, bringing an escalating risk of catastrophic real-world harm.

The framework also pushed for a balanced regulatory approach that allows government intervention without stifling innovation. Anthropic maintained that when an AI model presents extreme dangers, regulators must possess the statutory power to block it, or impose heavy fines tied to global revenue that multiply with subsequent offenses.

Until now, Anthropic has not openly endorsed any specific federal bill in Congress, though it has thrown its support behind several state-level AI safety laws, including a couple in California such as SB 53. The company publicly announced its endorsement of that measure, which was enacted last year and requires companies to create and share their safety frameworks while working to mitigate major risks. SB 53 also obliges companies to report any serious safety incident to the state within 15 days.

Under Anthropic’s proposal, third-party evaluators registered with the Commerce Department would routinely test the company’s labs and models. While labs are permitted to finance these assessments, the compensation cannot be contingent upon the evaluator’s findings. Furthermore, any discovery of an “imminent enumerated risk” would have to be reported to federal authorities within 72 hours of completion. If enacted, the proposal would extend Anthropic’s push for enforceable safety rules from state capitols to the federal level.

If the Commerce Department determines that a model poses a risk, it can ask a judge to block its release. The provision, however, prohibits any other US “department, agency, or officer” from stopping a model’s development, deployment, or internal use. Nonetheless, a federal judge in the District of Columbia retains the authority to order such a suspension.

FTC sends subpoenas to OpenAI and Anthropic

On Wednesday, the Federal Trade Commission sent civil subpoenas to OpenAI and Anthropic, requesting records on safety incidents that have become a growing crisis for the industry. OpenAI responded to the subpoena, saying, “As we improve our models and they increasingly help people in their day-to-day lives, safety remains a top priority. We will engage constructively with the FTC.”

A source revealed that the agency intends to extend its subpoena power to third-party entities in the near future. The regulator could force executives from Anthropic and OpenAI to testify publicly, giving lawmakers a platform to challenge the tech giants on safety as public fear of potential fallout grows.

According to a recent poll conducted by The Associated Press-NORC Center for Public Affairs Research, nearly two-thirds of US adults think AI is developing too fast. Eight in 10 respondents also stressed the importance of government efforts to keep the technology under human control. The findings sketch the public mood surrounding both the FTC’s inquiry and Anthropic’s push on Capitol Hill.

So far, President Trump has continued to reject calls for a slower pace of AI development, citing the technology’s importance to US strategic interests. He recently revealed a “Super Intelligence Force” to help align federal government efforts on AI while maintaining engagement with consumers, public interest organizations, and other stakeholders. Where Anthropic’s draft fits within that posture remains an open question.