Who Judges Matters: Measuring Family-Conditioned Preference in LLM-as-Judge Panels
Who the judge is can affect an LLM-as-judge result, but measuring that effect without confusing it with candidate quality is difficult. We study four open-weight families (Llama 3.1, Qwen 2.5, Gemma 2, and Yi 1.5) in a fully crossed pairwise design with 9,312 judgments. A common per-family statistic is strongly confoun...
David Ababio Awuni, Luke Achenie, Benjamin Tei Partey et al.
· 1 citation