Skip to content

JudgeProfile: Understanding and Steering Subjectivity in LLM Judges

Sep 2026 · 0 citations
Computer Science

Abstract

LLM judges are inherently subjective, often favoring different responses in pairwise comparison when neither option is objectively wrong. To study this subjectivity, we introduce JudgeProfile, a framework that dissects LLM evaluation into perception (how a judge compares two responses across specific attributes like clarity, correctness, and detail) and prioritization (how much each attribute influences the final choice). We curate SubjectiveSet, a dataset of 50,013 response pairs from 17 public data sources, evaluated by 21 LLM judges across 87 attributes. We find a hidden consensus in perception: judges frequently agree on attribute judgments even when their overall choices diverge. Building on this separation, we first characterize each judge's prioritization using attribute weights estimated from its own overall choices. These weights differ across judges even when estimated from the same attribute judgments. We then learn new weights from reference labels to adapt their decisions to a target evaluation standard. Reweighting perceived attributes improves average held-out agreement with reference labels from 66.48% to 71.97%, outperforming fine-tuning and rubric prompting. Our findings show that understanding and steering the subjectivity of LLM judges requires attention not only to what they perceive, but also to how they prioritize it.

View source

Similar papers

Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration

LLMs are increasingly used as automated judges for model training and evaluation, yet individual judges exhibit systematic biases that undermine reliability. Much of prior work has studied biases in pairwise LLM-as-a-judge settings; in this paper, we focus on absolute scoring tasks, which mirror more realistic use case...

Gemma Zhang, Prachi Badarayani, Asmi Kumar et al. · 0 citations
Preprint Sep 2026

OSCAR: Order-aware Sensitivity Correction for AI Rankings

LLM judges evaluate candidate language models through pairwise comparisons of their responses. Judge-aware Bradley--Terry models aggregate pairwise comparisons by jointly estimating candidate scores and judge-specific sensitivities (how strongly each judge responds to quality differences), often interpreted as reliabil...

You Liu, Yue Liu, Quan-Chao Lu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Agreement Overstates Evidence: Error Dependence in LLM Judge Consensus

Consensus among LLM judges is often taken as strong evidence that a decision is correct. This assumes that judges make their errors independently. In practice, LLM judges are often trained and evaluated in similar ways, so they can make the same mistakes. We study how this dependency affects the reliability of consensu...

Elias Hossain, Niloofar Yousefi, Ser-Nam Lim · 1 citation
Open access Oct 2026

Agreement by Construction: A Deterministic Filter Inflates Inter-Judge Reliability in LLM-Judge Evaluation

Aim: This study investigates how a deterministic pre-judgment rule, which assigns a verdict before LLM judges evaluate a response, affects estimates of inter-rater reliability, with particular attention to Cohen’s κ. Methods: Using 600 paired judgments from the LoCoMo benchmark, we compared Cohen’s κ between two LLM j...

Akshaya Kumar · 0 citations
Preprint Oct 2026

BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments

AI evaluation pipelines often produce conflicting judgments rather than clean labels. In pairwise LLM evaluation, this conflict is especially visible: disagreement can arise from ambiguous items, underspecified rubrics, heterogeneous or unstable human raters, or an LLM judge whose verdict changes when the response orde...

Jia-Hao Zhang, Peng-Bin Feng, Chun-Lei Meng et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.