Weak-to-Strong Generalization via Direct On-Policy Distillation

Researchers demonstrated that weak models can match stronger model performance through on-policy distillation—training on outputs the weak model generates itself, corrected by a stronger teacher model. The approach avoids requiring access to internal representations or logits from the teacher, operating only on final predictions.

This method reduces the infrastructure required for knowledge transfer. Operators can deploy weak models as standalone inference endpoints without maintaining architecture compatibility with teacher models or exposing model internals for distillation. It also lowers the computational cost of scaling: smaller models can be trained more cheaply while maintaining capability parity with larger variants.

For builders, this shifts the economics of model deployment. Rather than running inference on larger models, teams can train smaller variants against stronger external models (including commercial APIs) and achieve comparable output quality. This particularly advantages cost-constrained deployments where repeated access to larger models is cheaper than continuous inference, enabling staged rollouts where weak models serve high-volume requests while gradually absorbing teacher knowledge.