SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation
Reinforcement learning with verifiable rewards (RLVR) has become a standard paradigm for improving large language models (LLMs) on various tasks, yet its sparse outcome rewards lack token-level credit assignment for intermediate steps. To address this, on-policy self-distillation (OPSD) leverages a self-teacher with pr...
Zhenrui Yue, Hui-Min Zeng, Yue-Qi Wang et al.
· 0 citations