On-policy distillation (OPD) is increasingly applied to frontier foundation model post-training. Prior work in this area has largely focused on algorithmic advances, yet it remains unclear how much OPD depends on its training questions and, in particular, how far this dependence can be reduced. Across two representativ...
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· 0 citations
MAS-OPD is presented, where Role-Advantage Specialization defines the role advantage as the difference between the teacher signals under target and non-target role conditions, and Privileged Attribution for Coordination attributes an interaction conflict to its source and supplies it to the teacher alone as privileged...
Qi-Yong Zhong, Mao Zheng, Ming-Yang Song et al.· 0 citations
USA is proposed, which converts per-parameter update magnitudes measured during a brief warm-up into per-coordinate perturbation radii, reducing curvature precisely on the coordinates that carry most of the merging displacement.
Qi-Yong Zhong, Mao Zheng, Ming-Yang Song et al.· 1 citation
The Alpha-Stabler framework is proposed, a plug-and-play framework with a Predictor that monitors principal-subspace intrusion for early collapse warnings, and a Controller that removes the principal-subspace component of activation gradients during backpropagation while preserving the orthogonal complement.
Yu-Chen Cai, Ding Cao, Qi-Xiang Yin et al.· 1 citation
Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout and schedules the teacher's intervention probability along a curriculum that decays to zero is proposed.
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· arXiv.org· 24 citations· ⚡3
Experiments on reasoning, code-generation, scientific-knowledge, scientific-knowledge, and tool-use benchmarks show that these implementations can be executed through the same verl-based backend while retaining their method-specific objectives and task-dependent performance profiles.
Jie Sun, Mao Zheng, Mingyang Song et al.· arXiv.org· 0 citations
Self-Evaluation Elicitation (SEE) is introduced, a method that surfaces a latent ability to predict how a judge will score its own output through a short cycle comprising a calibration-coupled reinforcement learning phase that improves the answer and predicts the judge, followed by a masked distillation phase that shar...
