Preprint
Aug 2026
SPOT: Sparse Probing and Outcome Calibration for On-Policy Distillation
On-policy distillation (OPD) provides dense teacher supervision on student-generated trajectories, but standard reverse-KL training can assign insufficient probability to other plausible continuations. Teacher entropy alone does not reveal whether uncertainty is concentrated among a few plausible next tokens or dispers...
Zi-Kun Qu, Min Zhang, Ming-Ze Kong et al.
· 4 citations