Jul 2026
SAF-OPD: Stable Advantage Fusion for On-Policy Distillation
SA is proposed, a Stable Advantage Fusion framework that avoids entropy collapse and consistently outperforms fixed-coefficient GRPO+OPD fusion, improving the aggregate score by 0.70% across all six model-domain settings while achieving more stable training.
Yifan Ding, Xin Wei, Yoshua Y. Li et al.
· arXiv.org · 3 citations