Stabilizing Policy Optimization via Logits Convexity
This work investigates the stability gap between SFT and RL from a gradient-based perspective, and shows that the convexity of the SFT loss with respect to model logits plays a key role in enabling stable training.