On-Policy Self-Distillation with Teacher Evolution
This paper proposes Teacher-Evolution Self-Distillation (TESD), which enables the self-teacher to generate rollouts and receive environmental feedback based on task correctness and context independence and performs teacher-oriented optimization to continuously enhance the model's self-teaching capability during distill...