FACTOR is introduced, which uses checkpoint-calibrated TD residuals to assign per-action credits that telescope to the trajectory advantage, and feedback-conditioned teacher-student likelihood gaps to allocate each credit across the realized action tokens.
Li-Chao Ma, Yang Sun, Shuai Zhao et al.· 3 citations
It is shown that policy entropy bounds both the policy gradient and probability update norms; consequently, entropy collapse effectively stops reward signal backpropagation, preventing further policy learning regardless of data quality.
Cong Qin, Jiaye Lin, Xiaoliang Fu et al.· Proceedings of the 32nd ACM...· 0 citations
Reinforcement Learning with Verifiable Rewards (RLVR) has become a standard approach for improving complex reasoning in Large Language Models (LLMs). However, entropy collapse remains a critical bottleneck, often leading to performance degradation. While typically attributed to reduced exploration, we identify a more s...
Cong Qin, Jiaye Lin, Xiaoliang Fu et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
