TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment
Reasoning distillation from powerful teacher models to smaller students faces the Gap Curse: as teachers grow more sophisticated, their complex distributions increasingly diverge from what students can approximate, causing performance degradation. Existing mitigation strategies either filter out challenging examples th...
Zhen-Yu Lei, Zi-Han Chen, Yao-Chen Zhu et al.
· 0 citations