Sparse RL Relaxation for LLM Reasoning
This paper proposes a sparse relaxation method for RL (called SpaRLax), which leverages the differentiability of soft-CoT reasoning over time steps and makes training more stable and effective, and adopts an entmax sparsity method that adaptively determines the degree of softness.