Preprint
Oct 2026
Beyond the Teacher: Understanding and Preserving Distillation Overshoot in Agentic On-Policy Distillation
On-policy distillation (OPD) trains agents on their own trajectories using supervision from a stronger teacher. We uncover a counterintuitive phenomenon in multi-turn agentic tasks: a smaller, initially weaker student can substantially outperform its teacher during training, only to regress as distillation continues. W...
Ji-Ke Zhong, Ming Li, Yu-Xiang Lai et al.
· 0 citations