When to Follow the Teacher: Advantage-Based On-Policy Distillation
Online Policy Distillation (OPD) often outperforms offline methods such as Supervised Fine-Tuning (SFT) by training on trajectories sampled directly from the student. However, standard OPD trains the student to follow the teacher at every state of the trajectory, assuming that teacher guidance is always beneficial. We...
Mahdi Sabbaghi, George Pappas, H. Hassani et al.
· 0 citations