The model that broke out of a lab will get the headlines. The millions of agents already inside your banking app, wreaking havoc, are the ones that should worry you more, says Rory Blundell.

Frontier AI labs are racing to build faster, more capable, more autonomous agents. This week showed what that race can cost. OpenAI ran an internal test with its models’ usual safety checks deliberately dialled down to see how effective they were at hacking. The models found and exploited a previously unknown flaw in the surrounding software, got online, and then broke into Hugging Face, the company hosting much of the world’s open-source AI.

That should alarm you. It should also make one thing clear: safeguards are not keeping pace with capability.

But it is not these highly capable AI agents that worry me most. It is the millions of far less capable agents already holding the keys to important systems, with almost no oversight.

Gravitee’s own research puts the number of AI agents now deployed in business at more than seven million. These agents are doing everything from running a local plumber’s Instagram account to managing the customer database for a multinational logistics firm, giving them access to millions of sometimes highly personal data points.

Earlier this year, thousands of people deployed always-on personal AI assistants through the platform OpenClaw. Many of those agents are still running now, quietly, with nobody checking exactly what they are doing.

AI agents do deliver real productivity gains. But most of them lack basic governance. Many run fully autonomously, with no human in the loop to approve their actions. That matters because the practical risk is not limited to dramatic breaches; it also includes routine systems that can touch calendars, code, payments, and customer records without clear oversight.

Chaos

Without those guardrails, failures are already happening. I hear stories of this chaos every day: agents that self-replicate, delete code, leak customer data, or go on an unauthorised spending spree.

One CEO recently told me how an AI agent that was supposed to manage team diaries decided the easiest way to clear some space was to delete every event in everyone’s calendar across the whole business. This is happening at scale, right now.

The UK’s AI Security Institute (AISI) is now investigating the OpenAI breach. That is good. But the government must not let the headline-grabbing case distract from where the real exposure sits.

Frontier models are often compared with nuclear weapons: rare, powerful and tightly controlled. The agent population is the opposite: millions of them, built and deployed by almost anyone, with almost none of them tracked.

Current attempts at regulation miss this point. The EU’s AI Act sorts systems into “low risk” and “high risk,” as if that risk were fixed. It is not. An agent that is low risk one day can become high risk the next, when it gains new access or new capability.

Regulation must start with accountability. Just as every employee has a manager, every AI agent needs a named human owner. We cannot let autonomous software move money, access data, make decisions and take action with all the power of an employee but none of the accountability.

Firms that are waking up to the risk are using platforms like ours, which provide a central control panel that watches, secures and manages what AI agents do, who they talk to and which tools they are allowed to touch. As adoption spreads, the question for businesses is less about whether to use agents than how to keep them visible, attributable and constrained.

The model that broke out of a lab will get the headlines. The millions of agents already inside your banking app, wreaking havoc, are the ones that should worry you more.

Rory Blundell is CEO of Gravitee, a software firm that helps enterprise companies govern and secure their AI agents.