Calibrating Uncertainty with Cross-Model Consistency for LLM Hallucination Mitigation

Abstract

Large Language Models (LLMs) are known to hallucinate, generating non-factual outputs that undermine user trust. Recent ensemble-based approaches leverage uncertainty estimation to select among multiple LLM responses, achieving promising results in hallucination mitigation. However, these methods treat each model's uncertainty independently, overlooking a crucial signal: cross-model consistency. In this work, we observe that answers agreed upon by multiple models are significantly more likely to be correct-a manifestation of the "wisdom of crowds" principle. Leveraging this insight, we propose Consistency-Calibrated Uncertainty Fusion (CCUF), a framework that calibrates individual model uncertainties using cross-model consistency scores. When multiple models converge on the same answer, CCUF reduces the associated uncertainty estimate; when answers diverge, uncertainty remains elevated. This calibration mechanism enables more reliable answer selection for factoid question answering. Extensive experiments on TruthfulQA, TriviaQA, and FACTOR-news benchmarks demonstrate that CCUF consistently outperforms state-of-the-art hallucination mitigation methods, surpassing the previous best ensemble method UAF by 3.4% in accuracy while exceeding GPT-4 performance on TruthfulQA by 5.2%.