Kuaishou’s KwaiKAT Team has introduced KAT-Coder-V2.5, a coding model designed to work inside real, executable software repositories rather than produce single-turn code snippets. The served model is available through StreamLake. A separate open-weight variant, KAT-Coder-V2.5-Dev, has been released on Hugging Face under the Apache-2.0 license.

The release is centered on agentic coding workflows, where a model must inspect a repository, understand a task, edit files, run tests, and verify whether a patch is correct. That focus reflects a broader shift in coding-model evaluation from isolated programming prompts toward repository-level software maintenance, where reproducible environments and reliable test feedback can be as important as code generation quality. The project emphasizes repository environments, data construction, sandbox reliability, reinforcement learning infrastructure, and benchmark evaluation.

AutoBuilder builds environments that run the intended tests

The research defines a verifiable coding task as a triplet: a precise task description, an executable repository environment, and a set of validation tests. A patch is considered correct only if it passes the full validation set.

Tasks are mined from real pull requests and commits, following the SWE-bench lineage. The merged code change provides a golden patch, while the accompanying test change provides a test patch. The system does not use raw issue text as the specification. Instead, task descriptions are regenerated into three components: a problem statement grounded in the golden patch, requirements derived from the test patch, and interface constraints inferred from both sources. A clarity check removes tasks that are ambiguous, incomplete, underspecified, or internally inconsistent.

AutoBuilder is responsible for the environment construction process. A build agent examines the repository and writes a configuration script that installs dependencies and runs tests from a clean checkout. A verification agent then executes the script in an isolated sandbox.

The acceptance process does not rely on exit codes or log-pattern matching. Instead, verification parses structured output from test frameworks. An environment is accepted only when more than 90% of expected tests are collected and pass/fail outcomes are reproducible across runs. Failures are returned as structured information for iterative repair.

By combining a preconfigured base environment, build-system templates, and a retrievable library of distilled build recipes, the team increased the environment construction success rate from 16.5% to 57.2%. The resulting dataset includes more than 100,000 verifiable environments across 12 programming languages. Git history, commit metadata, and other exploitable traces are removed so agents cannot obtain the reference solution directly from the repository.

Data Scaling Flywheel filters for process quality

The project argues that filtering trajectories only by final test success can be misleading. Some passing runs may depend on hard-coding, bypassing intended mechanisms, or taking shortcuts tailored to tests. At the same time, some failing runs may still contain useful search, localization, and repair behavior.

KwaiKAT addresses both cases. For near-miss attempts, targeted process-level hints identify what should be inspected or verified without revealing the solution. This raises the pass rate for previously zero-pass tasks to roughly 20%. Because hinted trajectories contain information that would not be available at inference time, the verified patch is then fixed and a hint-free trajectory is regenerated from the original task context. Only samples that pass verification, contain no hint leakage, and remain consistent with the patch are retained.

For trajectories that already pass, rule-based gates remove invalid, unstable, or exploitative examples. A scoring stage then evaluates exploration, localization, pre-edit reasoning, specification fidelity, adherence to repository conventions, patch minimality, verification quality, recovery behavior, and honesty.

A third mechanism is aimed at reducing harness overfitting. Tool names, argument conventions, output formats, and prompt templates are randomized while keeping functionality unchanged. Because verification is tied to test outcomes rather than harness traces, the same task can be presented under multiple harness configurations. The system also injects realistic perturbations, including missing dependencies, transient command failures, truncated outputs, and noisy logs.

Sandbox failures affected rewards before algorithmic limits

During KAT-Coder-V2 training, slow reward curves were first attributed to the reinforcement learning algorithm. An audit later found that approximately 16% of trajectories failed because of sandbox infrastructure issues rather than model policy. Boundary misalignments sometimes emptied observations for about 40 steps and corrupted rewards.

The team implemented three infrastructure fixes. First, an early-release image eviction policy reduced disk usage from 95% to 60%, lowering timeout-induced invalid rollouts from 6–7% to under 1%. Second, correcting environment variables during remote sandbox initialization prevented system overrides that had flipped rewards on 6–7% of samples, reducing those errors below 1%. Third, the Gateway Server bypassed mainstream chat endpoints, which had caused 40% token drift at roughly a 200-turn scale by re-applying apply_chat_template and re-tokenizing. The system instead called /generate directly to maintain rollout token alignment.

Together, these changes reduced the sandbox feedback error rate from about 16% to below 2% and cut training collapses by an order of magnitude. The finding also highlights a practical constraint for agentic-code training: reward quality depends on the surrounding execution system, not only on model architecture or optimization method.

Asymmetric PPO and three-tier rewards

The researchers selected PPO with GAE instead of critic-free trajectory methods because production harnesses split sessions into structurally different samples, making group baselines more difficult.

The training setup uses an asymmetric actor–critic design. The Critic receives privileged training context, including rewards, tests, coverage, patches, metadata, and future turns. The Actor sees only the rollout state. The Critic and the additional context are discarded at inference.

Rewards are organized into three tiers. Core Task Scores require all fail_to_pass and pass_to_pass tests to pass. Standard Behavior Constraints penalize duplication, invalid tool calls, and debug remnants. Failed Trajectory Incentives score file retrieval through F2 and assign partial credit for tests.

Five experts are combined through Multi-Teacher On-Policy Distillation using reverse KL, an off-policy start, and drift-aware truncation from Prune-OPD.

Benchmark results

Under a unified Claude Code harness, KAT-Coder-V2.5 led its panel on PinchBench with a score of 94.9, ahead of Opus 4.8 at 93.5. It ranked second on SWE-Bench Pro with 65.2 versus 69.2, and second on the internal KAT Code Bench with 53.1 versus 57.3.

The model performed less strongly on Terminal-Bench 2.1, where it placed last with 60.7, behind GLM-5.1 at 61.8 and Opus 4.8 at 84.6. On SciCode, it scored 50.3, matching GLM-5.2. The mixed results make the harness and benchmark scope important to interpretation, since repository patching, terminal operation, and scientific coding tests measure different parts of an agentic coding system.

The open-weight KAT-Coder-V2.5-Dev is a separate 35B-total / 3B-active MoE model post-trained on Qwen3.6-35B-A3B using 127K SFT examples, followed by reinforcement learning. It was evaluated under a separate in-house protocol, so its results are not comparable with the main flagship benchmark table.

The project’s primary materials include the paper, the StreamLake product page, and the KAT-Coder-V2.5-Dev model weights on Hugging Face.