This work audits five LLM overseers on 19 compliance tasks, varying only trace detail and evidence labeling, finding that elaborate traces shift the decision criterion toward rejection, increasing false alarms in susceptible overseers.
Online communities often treat large support margins in public elections as strong mandates. We argue that such margins can overstate the independent scrutiny behind a decision. Using 198,275 free-text rationales from Wikipedia admin elections, we introduce vote-text divergence, a measure that flags a decisive vote pai...
The cross-domain benchmark and the evaluation framework for intelligent synthetic-user evidence is made available on request, so that teams can determine in advance when synthetic-user evidence is safe for decision support and when it is not.
Zihang Chen, Di Zhu, L. Zheng· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.