Pair Difficulty Matters: Rethinking Pairwise LLM-as-a-Judge Evaluation and Consistency
Judges should be assessed on rank-gap-conditional metrics, ideally against human rankings, and this argument is formalized and validated in a controlled simulation and on two human-rated corpora.
Bruno Brocai, M. Becker
· 1 citation