COPC: Coupled Off-Policy Correction for Asynchronous LLM Reinforcement Learning
COPC is introduced, an actor--critic method combining token-level ratio masking with two-sided clipped-ratio weighting of TD residuals for return and advantage estimation that achieves the highest reported performance on tool-integrated mathematical reasoning and search, outperforming the strongest reported asynchronou...
Zi-Cheng Hu, Zhi-Jian Zhou, Xuan Zhang et al.
· 0 citations