Sparse RL Relaxation for LLM Reasoning
This paper proposes a sparse relaxation method for RL (called SpaRLax), which leverages the differentiability of soft-CoT reasoning over time steps and makes training more stable and effective, and adopts an entmax sparsity method that adaptively determines the degree of softness.
Jia-Bin Fan, Matthias Lindemann, Zita Marinho et al.
· 0 citations