#artificial intelligence
May 2026
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
It is found that the optimal distillation context depends jointly on the student model's capacity and the target task, and that no single universally effective configuration emerges.
Mohammadreza Armandpour, Fatih Ilhan, David Harrison et al.
· arXiv.org · 25 citations
· ⚡1