When the leaders of America's top artificial intelligence labs warn that the very systems they are building could endanger humanity, an obvious question follows: what do they stand to gain?

The answer, according to experts, analysts and former government evaluators who spoke to The Associated Press, may lie partly in politics and partly in business. The companies appear to be courting public favor ahead of the U.S. midterm elections, while positioning themselves for the substantial sums their labs and investors stand to make in long-anticipated stock market listings — initial public offerings, in which companies sell shares to public investors for the first time — on a jittery market.

The chief executives of Anthropic and OpenAI recently declared that the nation's most advanced AI models are so powerful they are dangerous, arguing they must be regulated and independently tested before release. In a rare show of unity, the two leaders have sketched out alarming scenarios in essays, social media posts and speeches at the United Nations.

Along the way, they are shaping the conversation over how their technology should be controlled. With their rhetoric, the companies appear to be seeking public favor and setting the terms for their own safety protocols in an unregulated market, the experts told the AP.

A resignation becomes a messaging opportunity

Those aims may not exactly align with what Jacob Coxon, an Anthropic engineer, had in mind when he quit this month via a post on X, calling for a pause on technology development to keep "superhuman" systems — AI more capable than humans across most cognitive tasks — from eluding their makers' control. But the companies saw his post as an opportunity to spotlight their own safety efforts and cast themselves as cautious market leaders — just as they need fresh capital before going public on Wall Street.

President Donald Trump, for his part, has rejected the need for new AI regulations, dismissing talk of risks to humanity as a "HOAX" designed to help China.

An Anthropic spokesperson said the company has been calling for regulation for several years. OpenAI spokesperson Liz Bourgeois noted that the company recently paused training of its most advanced models.

"People want to know AI is being developed safely, and that starts with what companies like ours do ourselves," Bourgeois said.

Shifting the spotlight

Sarah Shoker, who previously led OpenAI's geopolitics team, said steering the conversation toward unproven threats — and away from polarizing issues such as data centers' environmental impacts, uncontrolled hacking incidents, mass AI-powered surveillance and the use of AI systems in warfare — leaves Silicon Valley in a more comfortable position.

"Once again we're talking about existential risk, while deprioritizing a number of other safety-critical risks that exist today," said Shoker, now a senior non-resident fellow at the University of California, Berkeley Risk & Security Lab. "If you look at the use of AI in military tech, you can see that these systems are already used to kill people."

As AI companies have moved from building chatbots to advanced "world models" with 3D awareness — systems designed to simulate physical environments rather than simply generate text — debate has raged over how the technologies should be tested. A series of problematic incidents has shown leading AI companies failing to police themselves or design safe experiments. In recent months, AI agents from top labs have hacked into external websites after being allowed to escape company training sandboxes, interacted with U.S. government websites in unexpected ways, and appeared to achieve a mathematical breakthrough only to face accusations of stealing mathematicians' work.

OpenAI representatives recently told reporters that the company has talked with AI firms including Anthropic and Google about pausing development. They also argue that independent auditors are needed to check the labs' progress if the government will not regulate. Whether any coordinated pause materializes — and on whose terms — remains unresolved.

A skeptical White House and a little-known testing agency

Trump has shown little interest in regulating AI, which has been a major driver of recent U.S. economic growth. The Republican president has personally struck deals with Silicon Valley firms that tie the economy more closely to their successes. San Francisco venture capitalist Sacks, who co-chairs Trump's Council of Advisors on Science and Technology, has dismissed calls for an AI slowdown as fearmongering from the "Doomer Industrial Complex."

In reality, the Trump administration already evaluates some of the tech giants' models through a little-known federal agency, the U.S. Center for AI Standards and Innovation (CAISI). Created in 2023 under Democratic President Joe Biden, the agency was heralded as a clearinghouse where labs could voluntarily submit advanced AI models for testing.

The agency still exists, but the field has since expanded to include an array of independent evaluation firms, such as METR, a nonprofit based in Berkeley, California. In a recent essay, Anthropic CEO Dario Amodei suggested METR could help vet his company's safety practices.

Still, unlike regulated sectors such as restaurants, financial services or aviation, there are no universal standards for testing the safety and security of AI systems, said Andrew Strait, who recently left the United Kingdom's AI Security Institute, which worked with CAISI.

Notably, the companies are not calling for more oversight from the government agency equipped to handle such requests, said Conrad Stosz, who previously led CAISI. Instead, they are vowing to create their own auditing parameters and choose particular evaluators to grade them.

Stosz now works at one such evaluation lab that has tested systems for Anthropic, OpenAI and Google, and chairs the AI Evaluator Forum, which is drafting best practices for the field. He said even the forum has questions about what Amodei and OpenAI CEO Sam Altman want.

"Lots of evaluators are interested in embedding with labs and getting greater access, but it's a little ambiguous what embedded evaluators means," said Stosz, now head of governance at Transluce, which revealed last week that OpenAI agents hacked U.S. and Australian government websites. "Will evaluators be able to thoroughly investigate, assuming that access is granted in a way that does not undermine their independence and credibility?"

'A wall or a moat'

For Harrison Rolfes, a senior research analyst at PitchBook, the AI companies' calls for caution appear intended to curry favor with investors ahead of public offerings and the midterm elections, when the political winds could shift. Democratic governors are already rushing to show they take the warnings seriously — one reflection of how, in the absence of a comprehensive federal AI law, statehouses have become the most active arena for AI rulemaking in the United States.

Meanwhile, Rolfes said, the AI giants can block smaller competitors' growth by positioning themselves as the safest bet — not only for investors, but also for AI chipmakers and technology companies such as Nvidia and Google, which in turn benefit from deals that supply the giant AI companies with computing power.

"They're creating a wall or a moat within this sector. … It's genius and they're all going to make a lot of money," Rolfes said. "That's where I see this heading, having the top companies in the world just creating their own wall, and then using the safety as the reason."

Not all AI companies endorse a slowdown. Nvidia CEO Jensen Huang said in a phone conversation with Trump — a call he took onstage at a conference — that he agrees there has been excessive AI alarmism and that companies can choose to pace themselves.

Daniel Kokotajlo, who left OpenAI in 2024 over concerns similar to Coxon's, said he continues to worry that AI systems are advancing faster than companies can control. He fears existential scenarios such as AI supercharging the creation of bioweapons, nuclear war or other catastrophes.

"All of this talk is actually a way to sort of dissipate and redirect this political will that has built up, rather than actually channeling that political will to do something good," said Kokotajlo, who now leads an AI safety advocacy organization and said he counseled Coxon before his resignation. "Just please don't do the thing that's going to get us all killed."

This story was originally featured on Fortune.com.