Interactive-Policy Distillation with Bidirectional Propose-and-Verify
On-policy distillation (OPD) trains a student model on its self-generated trajectories with dense token-level teacher feedback. However, naive OPD may suffer from teacher unanchoring, where the student's reasoning trajectory drifts far from the teacher, causing the teacher to be queried on states it would hardly visit...
Shu-Tong Wu, Xi-Wen Chen, B. Rappazzo et al.
· 0 citations