Anthropic has lost another safety researcher as concerns grow inside the artificial intelligence industry over the speed of frontier-model development and the risks associated with increasingly capable systems.

Joe Tenton said he left Anthropic two weeks ago and will join METR, an independent group that tests risks in advanced AI systems. Tenton had planned to explain his decision later, but said Jacob’s resignation this week prompted him to speak sooner.

“I’d planned to write about that decision in more detail at some point, but Jacob’s resignation this week made me want to say more now,” Tenton wrote.

In a post explaining his move, Tenton said major AI laboratories are creating a level of danger that society has never faced. He argued that AI capabilities are improving at an extreme pace while companies are attempting to accelerate development even further.

The stated goal of many labs is to build systems that can improve AI research themselves and eventually reach “superintelligence.” Tenton warned that if those efforts succeed, technological progress could move beyond human control.

Tenton warns that AI labs are racing toward systems humans may not contain

Tenton said people could be living with AI agents more capable than every human within the next few years. He also warned that such systems could develop goals that differ from those of the people supervising them. If their capabilities become too strong to restrict, he said, the consequences could be disastrous.

“Humanity may not survive this transition,” Tenton wrote.

He said competition among frontier laboratories makes the problem worse. Any lab that slows its development risks falling behind another, creating pressure to spend less on safety than may be necessary.

Tenton pointed to several recent incidents. He said several hundred OpenAI agents were caught up in a hack that was somehow related to Hugging Face. Anthropic models also reportedly engaged in social engineering against people online.

According to Tenton, Anthropic has not experienced an incident as damaging as the one involving Hugging Face, although he attributed that partly to luck. If AI progress continues at its current rapid pace, he said, more dangerous incidents should be expected.

Tenton predicted that within the next few years, humanity could become powerless to manage the AI systems created during this period.

He also described the dilemma facing safety engineers at frontier companies. Resigning could allow more careless individuals to take their place, while remaining in the job could mean continuing to work on a system capable of causing immense damage.

Tenton said many former Anthropic colleagues are frightened by what they are building. He named Evan Hubinger, who managed him, and said Hubinger has estimated the chance of AI killing everyone at above 10%. Tenton added that Hubinger has worked on these questions for almost a decade, before large language models became a major business.

The CEOs of Anthropic, OpenAI and Google DeepMind, which belongs to Alphabet (NASDAQ: GOOGL, GOOG), have also backed a statement describing AI extinction risk as a global priority. Tenton said these concerns are common within the companies themselves. He added that humans are choosing to build the technology and can choose another path.

Tenton calls for independent AI checks as Trump emphasizes competition with China

Tenton said policymakers should consider managing the pace of AI capability development rather than allowing companies to accelerate without restraint.

“One question is whether we should actively manage the rate of capabilities progress, and if so, by how much. I think even holding AI progress at today’s pace, rather than the much faster pace the companies are aiming for, could be a win,” he said.

He identified political support and legal protection as major obstacles. Companies that jointly agreed to slow development could face antitrust scrutiny, Tenton said, making legal cover necessary if policymakers want competing laboratories to restrain themselves.

He also expressed doubt that governments will act while frontier development remains largely hidden from the public. A laboratory could experience a sudden intelligence jump or lose control of a system without outside observers knowing, he warned.

Tenton said his new role at METR will focus on independent testing. He wants external checks to become sufficiently common to influence companies’ incentives. He also called for stricter disclosure requirements, including disclosures about advances in AI capabilities, steps toward recursive improvement, accidents and near misses.

U.S. President Donald Trump has rejected warnings about AI extinction. When asked whether he was concerned that AI could wipe out humanity, Trump replied, “No, I don’t have any.”

Trump said his concern was winning the international AI race.

“I have concerns that if we don’t win AI, we’re going to be put in a very bad position,” he told reporters Thursday. “We are leading China right now by a pretty good period, I would say a year, which is, you know, considered a lot.”

The United States and China are competing for leadership in AI as Chinese models become more capable and attract users worldwide. Trump made the comments after researchers from frontier laboratories, including Anthropic and OpenAI, increased their public warnings about the pace of AI development.

Tenton’s explanation of his departure is available at Why I left Anthropic’s safety team.