Rethinking GRPO for Large Reasoning Model Safety Alignment
Beacon, comprising Safety Signal Recovery and Safety-Utility Decoupled Alignment, combines advantage-guided trajectory filtering and safe reference supervision to recover effective safety learning signals, while SUDA employs category-aware rewards and benign-only KL anchoring to mitigate over-refusal without compromisi...