Training one AI model to be good at everything is expensive. Salesforce AI Research says there may be a cheaper route: train several specialists first, then pour what they know into a single model.

In a preprint released on November 5, 2026, the team reports that training specialists separately and then merging them matched the performance of joint reinforcement learning on a combined math-and-code setup while using 42% fewer total training FLOPs.

What the Researchers Actually Tested

The paper is titled "Unpacking the Compute Efficiency of Strong-to-Weak OPD against GRPO" and is posted as arXiv:2611.08453.

GRPO stands for Group Relative Policy Optimization. It is a reinforcement learning technique that trains a model using verifiable outcome rewards, meaning the model gets credit when its result can be checked as correct. OPD stands for on-policy distillation. FLOPs, or floating-point operations, serve as the yardstick for training compute throughout the study — a yardstick that matters because it tracks closely with the time and infrastructure cost behind every training run.

The study pits these two approaches against each other on a simple scoreboard: how much held-out accuracy does each method deliver for every unit of training compute spent? Held-out tasks are problems excluded from training, so accuracy on them reflects generalization rather than memorization. To answer that, the researchers measured accuracy on held-out tasks against training FLOPs across multiple domains, including math and code, at different task difficulties and with different teacher setups.

The Headline Result: Specialists, Then a Merge

The standout finding involves what the paper calls multi-teacher OPD. Instead of training one model on math and code at the same time with GRPO, the researchers first trained separate GRPO "expert" checkpoints for each domain. They then distilled those experts into a single student model. Distillation here means training the student to reproduce the experts' outputs, carrying over their capability without rerunning the full RL process on the combined domains.

In the math-plus-code setting, that student model reached performance comparable to joint GRPO training, with a total compute bill 42% lower. The economics are the point: the reinforcement learning work was paid for once per specialist, while the merged student absorbed it through a distillation pass that the paper prices at a small fraction of that bill.

The merge step itself turned out to be cheap. According to the research, distillation accounted for just 5.2% of the cost of training the specialists in the two-domain case. When the setup expanded to four domains, distillation's share rose to 8.0% of specialist training FLOPs.

Where Distillation Fell Short

The paper does not crown distillation as a universal winner, however. When OPD relied on external post-trained teachers, meaning models trained elsewhere rather than in-house experts, it generally underperformed GRPO on a per-FLOP basis. Those external teachers also tended to make student responses longer than necessary.

There was a bright spot on the teacher front, though. Self-GRPO teachers, built from the student's own earlier checkpoints, proved highly efficient at recovering single-domain accuracy. The research summary describes reductions in distillation FLOPs of up to a factor of 50 in that setting.

Why the Result Matters for AI Training Budgets

The study fits a broader effort at Salesforce to make large language model post-training cheaper. It also builds on ongoing development of pipelines that use reinforcement learning with verifiable rewards, a family that includes GRPO. Post-training is the stage that shapes a model's behavior after its base training, and it is why accuracy-per-FLOP comparisons like this one draw attention from teams balancing capability goals against finite compute budgets.

The four-domain numbers hint at scalability. Distillation's share of cost rose from 5.2% to 8.0% as domains doubled, which suggests the merge step stays relatively affordable as the expert lineup grows, at least within the configurations tested.

There are caveats worth keeping in view. This is a preprint, so it has not yet been through peer review, and the headline 42% figure comes from a specific two-domain math-and-code comparison. The developments worth watching from here are correspondingly concrete: formal peer review, and whether the split-then-merge advantage holds in domains and configurations beyond the ones tested.

The original report was published by CryptoBriefing, where it was edited by Diego Almada Lopez.