Correcting or Continuing? What On-Policy Distillation Learns from Student Errors
On-policy distillation (OPD) trains a student on its own rollouts under a frozen teacher, mitigating the distribution mismatch of off-policy supervision. While prior work has primarily trained on problems a student can largely solve, we ask whether OPD can improve a student when training is restricted to problems it cu...
Shuo Yang, Zhe-Yu Zhang, Mahito Sugiyama et al.
· 0 citations