Spend Teacher Tokens Where They Matter: Success-Referenced On-Policy Distillation
On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher, but providing such supervision for every rollout requires substantial teacher computation. We introduce Success-Referenced On-Policy Distillation (SR-OPD), which reduces this cost by selecting which promp...
Xiang-Yuan Chen, Fu-Tao Su, Kong Wang et al.
· 0 citations