Reinforcement learning has become the foundational paradigm for enhancing the reasoning capabilities of LLMs. Yet the dominant approach, Proximal Policy Optimization (PPO), relies on a geometrically degenerate clipping proxy that cannot adapt its resistance to policy shifts, leading to premature clipping and degraded e...
Zhi-Cheng Cai, Xin-Yuan Guo, Han Wu et al.· 0 citations
Direct On-Policy Distillation (Direct-OPD) is proposed, which transfers the teacher's RL-induced policy shift instead of running sparse-reward RL on the target model and consistently leverages weaker teachers to improve stronger target models.
Shiyuan Feng, Huan Gao, Haohan Chi et al.· arXiv.org· 8 citations· ⚡1
Subspace-Aligned Rewiring (SAR) shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.
Zhilong Zhang, Hongli Yu, Huan Gao et al.· 0 citations
Open-MOPD, a principled framework incorporating token-share balancing, gap-aware dynamic budget allocation, and student reward refresh, systematically restore cross-domain balance, elevating headroom recovery from 35.6% to 83.4% in a single deployable student.
Huan Gao, Haohan Chi, Yong Yan et al.· 12 citations· ⚡6
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.