Professional analysts adopted their AI agent's hidden, randomly assigned opinion on 86% of key judgment calls, against roughly 50% expected by chance, according to an experiment published Tuesday by Anthropic's Alignment team. Only 15% of participants credited the agent afterward, and none suspected its preferences had been planted.

The findings challenge the "human in the loop" arrangement widely used in AI-assisted work, which assumes the person who signs off on an AI's output is the one making the important decisions—a safeguard that carries particular weight in regulated sectors such as finance and insurance. Anthropic launched 10 agent templates for financial firms and insurers in May, stating that users have the final word—but the study suggests that division of authority may be less clear-cut than it appears.

AI agents—programs that carry out tasks on their own—are increasingly being handed expert work on the usual assumption that humans still make the decisions that matter. To test that assumption, Anthropic recruited 50 analysts experienced in causal inference, meaning working out from data whether one thing causes another. Each analyst used Claude Code, Anthropic's coding agent, to advise a fictional farming cooperative on expanding an advisory program.

Unknown to the participants, a coin flip determined which option each agent favored. That preference was hidden in the system prompt—the instructions that shape how an AI behaves. The same instruction layer is where companies typically set the rules for agents they deploy, which is why the finding speaks to real deployments, not just the lab.

"Across the four analytical choices, analysts went with their agent's lean 86% of the time. If the agent's input made no difference, we'd expect about 50%," Anthropic said.

The pull was strongest among analysts who had committed to an answer before the agent weighed in: they abandoned their choice 60% of the time when the agent disagreed, compared with 6% when it agreed—though that figure rests on only 38 cases.

In a post-task survey of 48 analysts, 77% repeated the agent's arguments in at least one answer, sometimes word for word. Only 15% credited the agent, and 52% described the final recommendation as more their own than the agent's.

"For individual experts, this could mean losing agency over their own work, a form of what researchers call disempowerment," Anthropic said, describing a dynamic in which a human voluntarily gives up control over their own decisions. "The work still goes out under the expert's name," the authors wrote, "but the decisions that most depend on their judgment are in reality settled by the agent's prior beliefs."

The agents in the study were instructed never to mislead the analysts. Even so, they rarely argued for the alternative to their assigned preference. "With only one side argued well," the authors wrote, "the analysts could adopt the agent's assumptions without working out their own."

That one-sided advocacy echoes earlier concerns about AI agreeableness. In April 2025, OpenAI rolled back an update to ChatGPT that it called overly flattering or agreeable.

Anthropic also noted that error checks struggle to catch the problem, because judgment calls have no single correct answer—the finished work looks sound whichever option is picked.

The company stressed that the findings are not an argument for using AI less. Analysts working with the agent finished faster and made far fewer serious errors than those without, though the no-agent comparison group was small and less experienced. As a remedy, Anthropic suggests designing agents that argue the best case for each option, or that ask for the expert's view first—though either approach could overwhelm users with information. Whether such designs make their way into deployed products, including Anthropic's own agent templates, remains to be seen.

The researchers cautioned that the agents' preferences were deliberately amplified for the experiment, and that it covered a single task, field, and model—so real-world effects may be smaller. Measuring deference outside the laboratory is the hard part: the team could detect it only because it randomized each agent's lean, something the authors say is rarely possible in real work.