PROOF-Gen: From Optimized Data to Better Distillation
Apple's PROOF-Gen turns teacher model failures into distillation gold — cutting frontier costs while scaling tool-calling agents.
Why it matters
A new approach to model distillation that learns from failures (not just successes) could reshape the economics of deploying tool-calling agents at scale. Instead of discarding the 57% of failed teacher trajectories, PROOF-Gen extracts signal from near-misses, reducing the cost and frequency of frontier-model calls in production post-training loops.
The key facts
6 to knowStandard distillation generates-and-filters: keeps passing trajectories, discards failures
57% failure rate on τ-bench; two-thirds are near-misses (most tool calls correct, but incomplete)
PROOF-Gen mines hard scenarios from failures instead of re-running on identical failure patterns
Targets tool-calling capability distillation for deployable models
Posted to Apple Machine Learning Research (peer-reviewed or internal release)
Published August 26, 2026
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Supervised fine-tuning on teacher-generated trajectories is the standard first stage for distilling tool-calling capabilities into deployable models. Post-training pipelines that drive shipped tool-calling agents re-run this stage on a daily or weekly cadence, paying the frontier-teacher cost each…