Data-free On-policy Distillation
On-policy distillation (OPD) has become a standard component of frontier post-training pipelines, yet how much its training data actually contributes has gone largely unexamined. On the two teacher--student pairings most common in practice, we find OPD almost indifferent to its data: eight prompts already match a 17k-p...
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.
· 0 citations