CompassOPD: Cross-Family On-Policy Distillation via Within-Family Likelihood Shifts
On-policy distillation (OPD) provides dense token-level supervision on student-generated trajectories. Although OPD performs strongly when teacher and student belong to the same model family, we find that its effectiveness degrades in cross-family settings even after tokenizer alignment, with substantially stronger ext...
Nai-Bin Gu, Qing-Yi Si, Chen-Xu Yang et al.
· 0 citations