A joint evaluation by the United Kingdom's AI Security Institute (AISI) and the US Center of AI Standards and Innovation (CAISI) has found that Moonshot AI's Kimi K3 falls behind leading American AI systems in building software exploits and executing simulated network attacks. The institutes published their findings on July 23, ahead of Moonshot's planned release of the model's full open weights on July 27. The assessment is part of a broader pattern of government-backed safety institutes conducting pre-release evaluations of frontier models before they reach public availability.

According to the report, Kimi K3's safeguards did not prevent it from assisting with offensive cyber operations. The evaluators stated: "Kimi K3 is capable of autonomously attacking small, weakly defended, and vulnerable enterprise systems when directed to do so and given initial network access."

ExploitBench Results

The two institutes tested Kimi K3 using ExploitBench, a Carnegie Mellon-designed benchmark that measures how effectively a model can drive a software exploit to completion. The benchmark is built on 41 vulnerabilities discovered in Chrome's V8 engine after 2023. Benchmarks like ExploitBench have become a standard tool for AISI and comparable bodies to produce comparable, reproducible measurements of model risk.

Kimi K3 scored 32% on the benchmark. Leading US models averaged 76.2%, while China's GLM-5.2 scored 24%.

Arbitrary code execution — the benchmark's most severe outcome — grants an attacker full control over a target machine. US models achieved arbitrary code execution on an average of 20 out of 41 tasks. Kimi K3 achieved it on none. GLM-5.2 similarly failed to reach that threshold. While Kimi K3 currently leads open-weight rivals in general cyber capability, it falls short at the stage where an actual attack would cause the most damage.

Simulated Network Breach Test

The second evaluation, called "The Last Ones," places a model in a simulated corporate network containing approximately 20 hosts across four subnets. Completing the full 32-step intrusion path would take a human specialist roughly 20 hours.

Kimi K3 averaged reaching step 17. The most capable US models averaged 28.5 steps, and GLM-5.2 managed 11. However, evaluators noted that Kimi K3 completed the entire chain once in ten attempts, staying within the 100-million-token ceiling set for the test. The best-performing US models solved the full chain six or seven times out of ten.

The institutes interpreted Kimi K3's single successful run as evidence that the model possesses the underlying capability but cannot reliably reproduce it on demand. They also noted that the simulated environment lacked active defenders and included a built-in path to the target, conditions that inherently favor any attacker.

Safeguard Concerns and Open-Weight Risk

The evaluators highlighted the model's willingness to carry out offensive tasks without refusal as a significant risk. AISI has previously cautioned that increasing open-model capability creates "a persistent and irreversible risk of misuse." Once Kimi K3's weights become publicly available on July 27, the model's behavior can no longer be restricted by whoever hosts it. Unlike closed or API-only systems, where developers can update safeguards or revoke access, an open-weight release means any downstream user can strip guardrails and redeploy the model without oversight.

The full preliminary assessment is available on the AISI website.

Gap Between General and Cyber Performance

Prior to this report, Kimi K3 had demonstrated strong results on general benchmarks — a track record that remains unchanged. The 2.8-trillion-parameter Chinese model reportedly outperformed Anthropic's Claude 4.8 and OpenAI's GPT-5.5, and topped several leaderboards.

The institutes suggested this performance gap may stem from the model's training methodology. On July 22, White House science director Michael Kratsios accused Moonshot of distilling Anthropic's Fable model by using its outputs as training data. Distillation — the practice of using a more capable model's outputs to train another model — is common in the industry but can propagate both strengths and gaps from the source model. Anthropic's classifiers block advanced offensive cyber prompts, meaning a training dataset scraped from Claude responses would contain limited material on those specific skills. Kimi K3 lacks such classifiers, which allowed it to match Western models on general tasks while still underperforming in exploit development.