Alibaba has introduced QwenGyre, a reinforcement learning framework designed for what the company's research team calls "XLong-horizon" agent training: tasks in which a single rollout may run for hours, involve hundreds of interactions between the model and its environment, and consume close to one million tokens.

The framework responds to a shift in how LLM agents are deployed. As they graduate from resolving individual repository issues to constructing entire codebases, conventional online RL pipelines run into two obstacles. Rollout durations vary so widely that GPUs sit idle while a handful of straggler executions finish, and branching execution paths generate massive volumes of redundant training data. At horizons measured in hours and hundreds of thousands of tokens, both obstacles convert directly into wasted compute: idle hardware during rollout tails and duplicated tokens raise the cost of every training run.

Elastic Scheduler Targets GPU Underutilization

The first component, the Elastic Scheduler, addresses GPU underutilization. Existing frameworks typically do one of two things: alternate an entire GPU pool between rollout and training (colocate), leaving hardware idle during long rollout tails, or statically partition GPUs between the two stages (async), which prevents either side from borrowing the other's spare capacity. Each design trades one form of idle time for another — the colocate approach idles the whole pool during rollout tails, while the async approach locks spare capacity on whichever side cannot use it.

QwenGyre instead groups hardware into independently switchable "cells" that reallocate between inference and training as the volume of unfinished rollout work changes. Live harness executions are never interrupted: model requests are rerouted mid-flight to the remaining rollout engines, with KV caches migrated over RDMA (remote direct memory access) so that no computation must be repeated. Freed cells then join the in-progress training step as additional data-parallel replicas, pulling authoritative weights from a core cell, while streaming training distributes micro-batches according to each group's progress so that all cells finish in step.

Role transitions are fast. On average, switching from rollout to training takes 8.5 seconds, and switching back takes 3.5 seconds — negligible against rollout executions lasting hours.

Taming Branching Trajectories

The second component, the Trajectory Processor, handles the training data itself. At XLong horizons, agent harnesses compact saturated contexts, delegate subtasks to sub-agents, and retry failed paths, so one execution produces a branching tree of divergent context paths rather than a linear trajectory. Naively expanding every path duplicates shared prefixes and inflates training cost — a burden that grows with the near-million-token contexts these runs produce.

QwenGyre instead reconstructs the execution as a prefix-shared trajectory tree, preserving each model output's exact original conditioning context. Only tokens actually generated by the policy are trained on, and shared targets are counted once through on-tree masking.

Partial progress is also scored: executions that hit their timeout are still evaluated on their preserved workspace state, so partial attempts carry meaningful rewards rather than being discarded. Given that a single execution can hold hardware for hours, that recovery preserves signal that would otherwise be thrown away. At training admission, a bounded number of trajectories is selected per execution — capped by role priority, with the main agent first, then sub-agents, then summary paths — and token losses are averaged within each execution so that path count cannot skew the weighting.

Benchmark Results

Applied to Alibaba's flagship Qwen 3.8-2.4T model at roughly 700K tokens per rollout, QwenGyre raised the NL2RepoBench pass rate from 52.5% to 58.5% in 48 training steps, completing them in 75.4 hours versus 134.6 hours for the async baseline and 91.5 hours for the colocate baseline — speedups of up to 1.78× and 1.85× respectively, achieved while matching baseline training scores. Gains held consistently across all three evaluated workloads — NL2RepoBench, DeepSWE, and TerminalBench — under equal GPU budgets.

The nearly 60 hours reclaimed against the async baseline underscores why scheduling matters at this scale: shorter wall-clock time at score parity lowers the time cost of every training cycle. For readers weighing similar infrastructure claims, the details to note are the two conditions attached to these numbers — matched training scores and equal GPU budgets — and the fact that all figures come from rollouts of roughly 700K tokens, the XLong regime the framework is built for.

Source: Alibaba's QwenGyre Tackles The Million-Token Problem With Elastic GPU Scheduling For XLong RL via Metaverse Post.