Context quality is identified as a central bottleneck in on-policy self-distillation and the value of separating rollout-conditioned guidance from canonical supervision is demonstrated, demonstrating the value of separating rollout-conditioned guidance from canonical supervision.
Meilin Yang, Zixuan Ding, Jianhao Nie et al.· 0 citations
This paper presents a tutorial-style, collective-centric taxonomy for collective communication, which organizes recent advances into three layers: communication planning, which generates topology-aware collective schedules; communication execution and adaptation, which maps these schedules onto GPU runtimes and hardware in real clusters; and computation-communication coordination, which turns collective optimization into end-to-end training and inference benefits.
Xuebin Song, Menghao Zhang, Yue Liu et al.· IEEE Network· 0 citations