NewsMacroNearly 2 Million Canary Tokens Expose Prompt Data Leaks Across the AI Supply Chain

Nearly 2 Million Canary Tokens Expose Prompt Data Leaks Across the AI Supply Chain

Author: CryptoBriefing·

Key Takeaways

  • •HD Moore's study embedded nearly two million trackable canary tokens into AI prompts from July to November 2026, revealing extensive leaks across AI routers and inference services.
  • •Approximately 200 AWS credentials were exfiltrated in targeted attacks in late October 2026 and used by attackers across Amazon services including S3 and IAM, though the keys led only to controlled infrastructure.
  • •At least three AI hosting providers were compromised, with confirmed theft activity beginning as early as mid-July 2026, and none detected the breaches until Moore notified them in November 2026.
  • •Some AI resellers and marketplaces captured customer prompts through built-in logging despite zero-data-retention policies.
  • •A unique seeded URL surfaced in an internal chat at a Chinese company 24 days after submission, and prompts from US users were traced back to Chinese AI organizations.
Nearly 2 Million Canary Tokens Expose Prompt Data Leaks Across the AI Supply Chain

Security researcher HD Moore set out to answer a direct question: what happens to a secret once it is typed into an AI application? To find out, he planted roughly 2 million trackable "canary" credentials and URLs in prompts and pushed them through the network of services that powers modern AI tools.

The answer was not reassuring. The ecosystem leaked sensitive data extensively, and several of the companies doing the leaking had no idea it was happening. At least three AI hosting providers turned out to be compromised, and attackers walked off with customer prompt data.

Moore, a veteran researcher at Dragos best known as the creator of the Metasploit Framework — a penetration testing toolkit in wide use across the security industry — ran the study from July to November 2026. He embedded nearly two million unique, trackable canary tokens into requests sent to AI inference services and gateway routers, then monitored to see which ones would surface somewhere they should not. Plenty did.

How a canary catches a thief

A canary token works like a dye pack hidden in a bank bag. It looks like the real thing — an AWS access key or a private link, for example — and it phones home the moment someone tries to use it.

Canary tokens are a long-standing tool in defensive security, and that design makes the method unusually clean. A canary that fires means the data escaped and someone acted on it, not merely that it theoretically could have.

The test targeted a layer of the AI stack that most users never think about. Many AI applications do not talk to a model directly. Instead, they hand prompts to brokers such as OpenRouter or Vercel, often called "AI routers" or "AI gateways." These middlemen collect large volumes of prompts from end users, then forward each one to whichever inference engine fits their criteria, such as the lowest price.

What the tripwires found

The most striking result involved cloud credentials. Approximately 200 unique AWS credentials were exfiltrated during targeted attacks in late October 2026. Attackers then put the stolen keys to work across several Amazon services, including S3, Amazon's storage service, and IAM, which controls who can access what.

Fortunately, those keys led nowhere real. The attackers tested them in a controlled infrastructure environment, which is precisely what allowed Moore to observe them in action.

The thefts were not a one-off burst. The earliest confirmed theft activity dates back to mid-July 2026, meaning the leaks were underway almost as soon as the experiment began.

Outright compromise was only part of the picture. Moore also found evidence that AI resellers and marketplaces were capturing customer prompts through built-in logging features. That practice runs directly against zero-data-retention policies — the promise that a provider processes a request and then forgets it. The tripwires suggest some providers remembered anyway.

Geography told its own story. A unique URL seeded during the study showed up in an internal chat at a Chinese company 24 days after the original request. Moore also traced prompts submitted by US users back to Chinese AI organizations. Users sending a request through a router may have had no visibility into where it ultimately landed.

Perhaps the most uncomfortable finding was the awareness gap. The affected providers did not know any of this was happening; they learned about the breaches only when Moore notified them in November 2026. His report was publicly summarized on November 19, 2026.

Why the middle of the stack matters

The AI industry has spent considerable energy reassuring customers about what the major model developers do with their data. Moore's work points somewhere else: the brokers, resellers and hosting providers sitting between the user and the model.

Each extra hop adds risk. A prompt that passes through a router, a reseller and a third-party host has three chances to be logged, intercepted or stolen — and a privacy policy at the first stop says nothing about the third. Security teams have long focused on third-party software dependencies for similar reasons, and the findings suggest prompts traveling through AI intermediaries now carry a comparable exposure.

What this means

For companies building on AI services, the practical lesson is blunt: anything placed in a prompt should be treated as potentially exposed. That includes API keys, cloud credentials, internal links and customer records — material developers routinely paste into AI tools while debugging or automating work. The canaries show attackers are actively harvesting exactly that material and testing it quickly.

The findings also undercut the value of a zero-data-retention promise on its own. A policy is only as good as every party that touches the data. If a reseller's default logging captures prompts anyway, the guarantee becomes decorative.

The detection gap may be the biggest worry of all. Attackers stole data from at least three hosting providers over a period of months, and none of those providers noticed until an outside researcher told them. Whether the notified providers now disclose the incidents or change how their logging works remains to be seen.

Original coverage: Crypto Briefing