BiasReducer: Adaptive Bias Mitigation for Reward Models
Reward models score responses from large language models (LLMs) and guide LLM training toward human preferences. However, reward models can favor superficial attributes such as length or confidence, leading LLMs to produce higher-scoring but not more correct responses. Existing mitigation methods either retrain the rew...
Shuang Liu, Yongliang Miao, Yan-Guang Liu et al.
· 0 citations