Yu-Shu Li

Multi-Reward Hacking: From Aggregation-Induced Drift to Trust-Anchored Defense

To improve task performance while encouraging desirable response properties, LLM post-training often combines a primary reward (e.g., correctness) with auxiliary rewards (e.g., brevity or efficiency). However, how these rewards are aggregated can critically affect training outcomes. We show that symmetric aggregation,...