King Charles III has told the heads of the world's largest artificial intelligence companies that they must find ways to keep the technology under human control, warning that the very people who built it now fear it could develop “darker capacities” and even be turned to killing.

A private gathering in Scotland

The King delivered the remarks at Dumfries House in East Ayrshire, Scotland, at a gathering organized by the Ditchley Foundation. The event drew roughly 30 executives, officials, and researchers, among them Britain's AI minister Kanishka Narayan, Nvidia chief executive Jensen Huang, DeepMind co-founder Demis Hassabis, OpenAI finance chief Sarah Friar, and Anthropic's global affairs head Tino Cuéllar.

According to a private guest list viewed by Bloomberg, the invitation-only roster also reportedly included MI6 chief Blaise Metreweli, author Yuval Noah Harari, and Tsinghua University's Xue Lan. The breadth of that room — frontier-lab executives seated alongside a government minister, an intelligence chief, an author, and a Vatican adviser — showed how far AI safety has traveled from a technical debate to a question of statecraft.

Charles, 77, spoke for only about 20 minutes before leaving the room for what he hoped would be productive discussions. Those who value humanity's moral core, he said, are “anxiously seeking your reassurance that we will not lose control of our destiny, or our souls.”

“Intriguing and deeply concerning in equal measure”

The monarch described the technology's pace as “both intriguing and deeply concerning in equal measure.” He praised its record in medicine and the life sciences before posing a blunt question to his audience: “Surely, we need sufficient means of control before it is all too late?”

The King pointed to the concerns of AI's own creators, saying the systems risk “developing darker capacities, perhaps even to take life.” He urged the executives to weigh the “existential dangers” of such tools falling into the wrong hands.

It was not the first time Charles had voiced such concerns: in 2023, he sent a video message to the UK's AI Safety Summit at Bletchley Park expressing similar sentiments. Vatican AI adviser Paolo Benanti attended the Scotland summit, and Pope Leo XIV wrote in a May encyclical that AI must not remain in the hands “of a few.”

What the AI executives said

Demis Hassabis told delegates that artificial general intelligence—systems capable of matching or beating humans across many tasks—was “probably only a few years away.” He said such technology could have an impact ten times that of the Industrial Revolution. The odds of something going wrong, he added, were “definitely non-zero,” though he followed up by saying “the future is not yet written.”

Sarah Friar said the technology's growing power raised questions about keeping it in service of people, adding that “no one company or government can do that alone.”

Huang, whose Nvidia is valued near $5.15 trillion, told the room that safety was paramount and that a product should be held back if it was not ready. He argued for regulating AI products rather than the technology itself, and defended open models as a way to prevent smaller countries and researchers from being shut out. That distinction — rules for the technology itself versus rules for the products built on it — sits at the center of regulatory debates worldwide. Read alongside Friar's warning that no single actor can manage the technology alone, the exchanges sketched the two broad approaches now shaping AI oversight: where rules should attach, and who must carry them.

A debate sharpened by industry warnings

The summit took place against a backdrop of internal industry alarm, following a viral resignation post by Anthropic researcher Jacob Coxon and a prediction by his colleague Evan Hubinger that AI has a 10% chance of causing human extinction. Cryptopolitan has reported that Anthropic's Dario Amodei published an essay urging laboratories “pace the frontier,” and that OpenAI this week launched a system designed to disclose model misconduct after logging six incidents of concerning behavior.

Not everyone accepts that the danger is real, however: President Donald Trump has dismissed such fears as a “hoax.” How that gap closes — through lab-built safeguards like OpenAI's new disclosure system, product-focused regulation of the kind Huang endorsed, or the frontier “pacing” Amodei urged — is the question now following the executives out of Dumfries House.

Source: Cryptopolitan