Thinking Dangerously: How Reasoning Training Dilutes Safety in Language Models
Ask a reasoning model a harmful question and it refuses; ask it again after a long, entirely benign reasoning chain and it answers. Across six open-weight models, average refusal falls from 94.6% to 12.7% behind an 8,192-token benign prefix. We name this failure Safety Dilution and make it measurable: refusal decays sm...