When Sparse Reward Meets Dense Distillation: Training Dynamics of On-Policy Distillation
The cross-signal NTK is introduced, a token-level statistic that measures the alignment between reward and distillation gradients at position n and an empirical threshold beyond which naive mixing can lead to persistent training collapse is revealed.