As personal AI agents such as Muse and Claude become part of daily life, a growing number of people are handing them shopping tasks. But a new study from Visa suggests the technology is not yet up to the job: in a first-of-its-kind experiment, the payments giant instructed 22 different AI agents to purchase three everyday items—a television, a laptop, and a hair appliance—and the overall success rate came in at only around 57%.

The findings are published on a new Visa website that details the outcome of 15 separate shopping expeditions by each of the 22 agents—330 attempts in all. Perhaps unsurprisingly, agents powered by more cutting-edge AI models performed best. The Aragon 4 version of Google's Gemini and the Wonder version of Meta's Making notched the highest scores, each completing a successful purchase 82% of the time. A Claude agent placed third with a 76% success rate, while an OpenAI agent finished sixth at 64%.

Visa's published data charts the performance of all 22 models, plotting each agent's success rate on the vertical axis against the cost of running each shopping inquiry on the horizontal one. The company notes a clear spread tied to price: the five most expensive models succeeded on average 67% of the time, while the cheapest five scored 50%. For consumers weighing whether to delegate a purchase, the spread implies that results currently depend heavily on which model is powering the agent.

Part of the agents' struggle stems from a simple reality: the motivations behind any given purchase are often complicated, and in many cases consumers lack information and are not certain what they want.

Visa's experiment was designed to capture that dynamic. In the TV shopping tasks, the shoppers—who were themselves agents—were assigned a list of attributes they needed in a television, such as compatibility with a child's gaming console, a 30-day return window, and a model with Dolby Vision. As with real-world consumers, the purchaser would volunteer some information but not all of it.

The upshot was that, to succeed in Visa's experiment, an AI agent had to interact with the shopper in a way that would elicit the missing information in a reasonable time. Success meant buying a TV that matched the shopper's needs. In many cases, the agents instead bought the wrong product or failed to suggest anything suitable at all.

"We're not there yet from a trust or technology perspective. But we'll soon get into a world where consumers can trust and delegate," Visa's Global Head of Growth, Rubail Birwadker, told Fortune.

Birwadker noted that agent-driven shopping is still in its early days but advancing rapidly. He added that product discovery—the focus of Visa's experiment—is just one element of this new type of commerce that retailers and AI companies are working to figure out.

For AI agents to enter the shopping mainstream, companies will also have to deliver smooth and secure payments—a challenge that Visa and its rival Mastercard are already seeking to address by educating merchants and building tools to facilitate so-called agentic commerce, in which AI systems browse, select, and pay for goods on a customer's behalf.

There are also new and unresolved legal questions around AI agents going shopping. Most notably, it remains unclear who should bear liability when agents make rogue purchases or commit fraud. According to Birwadker, those questions will get resolved over time, in the same way an earlier generation of consumers and merchants learned to use e-commerce—though he added that agent-based shopping will be a very different experience from how people shop online today.

Alongside announcing the results of its agent shopping experiment, Visa also rolled out a website that allows consumers to interact with the agents used in its survey—a chance to test the agents' shopping skills firsthand.

This story was originally featured on Fortune.com.