RMEval: Bridging Reward Model Evaluation and Policy Evaluation through Ranking Consistency
This work proposes RMEval, a benchmark that revisits reward model evaluation through the lens of global ranking consistency, and evaluates existing reward models on RMEval and demonstrates that the metric exhibits a strong correlation with downstream policy performance, with Pearson coefficients exceeding 0.6 across mu...