Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
Apple researchers just revealed why your distillation strategy might be backfiring — and it's token-by-token.
Why it matters
Apple's research into on-policy distillation dynamics addresses a critical gap in reasoning model training: when teacher-student supervision helps vs. hurts. This matters for companies optimizing inference costs and model quality without massive retraining budgets.
The key facts
10 to knowResearch focus: on-policy distillation for reasoning models
Core question: optimal teacher model selection and self-distillation context
Apple's contribution: training-free methodology to evaluate distillation dynamics at token level
Problem solved: reduces need for costly aggregate training runs
Published by: Apple Machine Learning Research (Jul 2026)
On-policy distillation provides per-token supervision for reasoning models
Optimal teacher model choice varies token-by-token, not globally
Current approaches use costly training runs that obscure token-level dynamics
Apple introduces training-free method to evaluate distillation effectiveness
Research suggests self-distillation context selection is non-trivial
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions this signal is beneficial and under which it is detrimental. Which teacher model should be used, and in the case of self-distillation, which specific context…