A new research paper from Nvidia suggests AI agents may not need the ever-larger context windows the industry has been building toward, after finding that agents which compress their history into short handoff notes can match full-history performance on complex tasks at a fraction of the context size.

The study, titled "You don't need long context," shows that AI agents can preserve accuracy by regularly compacting their conversation history into concise handoff notes. Researchers were able to shrink the working context to as little as 16,000 tokens while still matching the performance of agents operating with their full history available. A model's context window—the text it can hold in working memory at once, measured in tokens—has become one of AI's most heavily advertised specs, with leading labs touting windows of a million tokens or more.

What the researchers tested

The paper was authored by Peter Belcak, Pavlo Molchanov, and nine co-authors. Its focus is frontier agents: AI systems designed to work through long, multi-step tasks rather than answer a single question.

Under the tested approach, instead of dragging the entire conversation forward at every step, the agent periodically condenses what has happened into a handoff note. Each subsequent step starts fresh with only that summary.

The team ran the approach on two models: Anthropic's Claude Opus 4.5 and OpenAI's GPT 5.6 Sol. Testing covered thirteen benchmarks, six of which were long-horizon tasks, including SWE-bench Verified and Terminal-Bench 2.1.

The numbers behind the claim

The headline result came from Claude Opus 4.5 on SWE-bench Verified, a software engineering benchmark. With one action per context—meaning the agent hands off after every single step—the model scored 70.2%. With its full history available, it scored 70.0%. In other words, the agent that kept forgetting everything except its notes did marginally better than the one that remembered it all.

The researchers also found that tuning the compaction settings could lift Opus 4.5's score on that task to as high as 74.8%.

Across configurations, the accuracy changes were generally modest. For Opus 4.5, shifts ranged from -3.7 to +5.6 points against the full-history baseline. For GPT 5.6 Sol, the range ran from -3.3 to +4.4 points.

In the single-action compaction tests, peak context dropped from 130K-200K tokens to between 15K-37K, while median context sat between 7K-9K tokens.

Why the context window race may be the wrong race

Nvidia's researchers argue that the more effective lever is how context gets managed, not how much of it a model can hold. The paper points to efficient summarization, selective reading of information, and structured handoffs as more effective ways to handle agentic workloads than simply stuffing more tokens into the window. That thesis runs against recent model marketing, but it has mechanical logic behind it: an agent re-reads whatever sits in its window on every step of a run, so each extra token of context is paid for again on every turn.

What this means for AI builders and their budgets

The most immediate implication is cost. According to the research, long-context inference can run up to 7x more expensive for models using 256K tokens compared with those using 16K tokens. Because mainstream API providers price by token, that gap lands directly on the operating bill for anyone running agents at scale. The study's scope—two frontier models across thirteen benchmarks—means its claims are strongest for the long-horizon software engineering and terminal tasks where they were tested.

Source: Crypto Briefing