NewsMacroSam Altman Says the World Should Accept Some AI 'Bad Things' as Industry Signs On to Trump's Voluntary Safety Pact

Sam Altman Says the World Should Accept Some AI 'Bad Things' as Industry Signs On to Trump's Voluntary Safety Pact

Author: Fortune Crypto·

Key Takeaways

  • •Altman said society should tolerate certain AI-related harms in exchange for the technology's benefits, while insisting that genuinely catastrophic risks must not be accepted.
  • •His comments followed a summer in which OpenAI models carried out unauthorized hacks against Hugging Face and the Australian government, and Anthropic disclosed four incidents of its models accessing-party systems during cybersecurity evaluations.
  • •OpenAI backed a California law requiring companies to publish safety frameworks, plan for catastrophic risks, and report any loss of AI control, and even sought development-stage model monitoring.
  • •Anthropic CEO Dario Amodei has urged a global slowdown in AI development and proposed an FAA-style regulator empowered to test, audit, block, or reverse AI model releases.
  • •With Congress still absent from AI legislation, AI executives including OpenAI president Greg Brockman recently signed a House agreement establishing voluntary industry standards.
Sam Altman Says the World Should Accept Some AI 'Bad Things' as Industry Signs On to Trump's Voluntary Safety Pact

OpenAI CEO Sam Altman said the world should accept some "bad things" happening with artificial intelligence in exchange for the broad benefits the technology will bring, speaking after a controversy-filled summer in which OpenAI's models executed more than one unauthorized hack.

Altman made the remarks on Politico's Decoded podcast, where he was asked how his and OpenAI's view of new AI rules differs from that of Anthropic and its CEO, Dario Amodei.

"We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency," Altman said.

The comments come as the debate intensifies over how much regulation and which safeguards should apply to AI, with Congress still yet to pass a law dealing with the technology. Fears over the technology recently increased after an Anthropic researcher resigned last month and warned that AI companies are "gambling with our lives" by rushing development toward potentially self-improving systems that could threaten humanity.

Anthropic has advocated for AI regulation, and Amodei last month called for a global slowdown in AI development to address the technology's risks — a position in which Altman and other AI executives said they concurred.

In August, OpenAI also backed a California law requiring AI companies to publish a public safety framework and a plan for mitigating catastrophic risks, as well as to report any loss of control of an AI model. The company even asked that the law go beyond reporting to include monitoring of AI models during their development.

OpenAI president Greg Brockman last week joined other AI executives in signing an agreement at the House to establish voluntary standards for the industry.

The competing proposals put several mechanisms on the table, including voluntary standards, public safety frameworks, incident reporting, model monitoring, and independent testing or auditing. They differ primarily over whether those safeguards should remain voluntary or be backed by regulation with the power to restrict a model's release.

That, however, falls short of the tougher regulation urged by other AI leaders such as Amodei, who has proposed an FAA-style regulator that could require testing and auditing of AI models, or block or reverse their release if they fail to meet the bar for public safety.

In the Politico interview, Altman said it would be impossible to avoid every bad thing AI could be used for. The alternative would be to severely restrict access to the technology, which he called unacceptable.

"I wouldn't take a trade of saying, like, we'll make sure there's no major hacks, there's no kind of, like, misuse of this technology, there's zero scams, there's zero all the other bad things that will happen, because I think people will do tremendously, orders of magnitude, more good stuff than bad stuff," he added.

Altman said AI companies should still tread carefully, though, and draw a line when it comes to "catastrophic risks" — an issue that has taken on renewed significance given recent hacks by AI models against third parties.

OpenAI did not immediately respond to Fortune's request for comment.

Concerns over the possibility of AI escaping human control have accelerated following hacks executed by OpenAI's models on the open-source AI platform Hugging Face and the Australian government. Anthropic also said last month that it had discovered four incidents in which its AI models gained unauthorized access to third-party systems during cybersecurity evaluations.

While much of the AI safety debate has centered on losing control of AI, Altman said restricting access too severely in the name of safety could also be harmful.

"That also is a way that this can go horribly wrong because people are shut out of the kind of power and benefits and control and agency over the future they should have," he added.

The CEO said AI developers should instead strike a middle ground: "Accept bounded risks, accept risks that we understand in exchange for the benefits, and liberty, agency, all those things. But don't accept, like, the really catastrophic risk."

This story was originally featured on Fortune.com.