CoRD: Coupled Rollouts Fill the Blind Spot of On-Policy Distillation
CoRD (coupled-rollout distillation) is introduced, which keeps the on-policy loss and moves only the distribution of training states toward the teacher, drawing rollouts from a geometric interpolation of student and teacher through the maximal coupling of speculative decoding.