Skip to content

When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

Sep 2026 · 0 citations · 34 references
Computer Science

TL;DR

This work introduces a two-stage keyword-perturbation method for hallucination detection and extends the same probabilistic framework to four hallucination regimes: knowledge deficit, wrong knowledge, context distraction, and unstable inference.

Abstract

Large language models can hallucinate even when the knowledge required for a correct answer is already available. We study this failure through a latent-key view of inference, where answer selection depends on competition among associations acquired during pretraining. We show that model predictions can be highly sensitive to individual query keywords, that these influential keywords exhibit entity-specific binding, and that their effects are systematically shaped by pretraining frequency. Multiple bindings can also compete and exhibit higher-order interactions within the same query. Based on this mechanism, we introduce a two-stage keyword-perturbation method for hallucination detection. By removing influential keywords and measuring how the model reorganizes its prediction, the method distinguishes errors caused by misleading key associations from correct decisions supported by diagnostic evidence. Across multiple models and benchmarks, perturbation provides a strong and transferable detection signal, reaching $0.910$ AUROC on probe-known ScientistQA. Finally, we extend the same probabilistic framework to four hallucination regimes: knowledge deficit, wrong knowledge, context distraction, and unstable inference. Their operational distributions across benchmarks provide diagnostic context for why different detector families succeed in different settings.

View source

Similar papers

The Alignment Paradox: How Post-Training Amplifies Confident Hallucinations in Language Models

Bounded margins mitigate confident hallucinations during post-training, implemented through an entropy-dependent margin bound in direct preference optimization (DPO) and shown to mitigate confident hallucinations during post-training.

Qing-Jia Huang, Ya-Kai Li, Jian-Guo Wu et al. · 0 citations
#machine learning Preprint Sep 2026

Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models

It is shown that epistemic uncertainty leaves a measurable trace within intermediate layers, where spikes in attention entropy are associated with reasoning breakdowns, and that attention dispersion is a promising complement to traditional hallucination detection methods.

S. More, Tanuja S. Pawar · 0 citations
#artificial intelligence Preprint Sep 2026

Domain-Specific Hallucination Detection in Large Language Models

A multi-signal detection pipeline combining fine-tuned DeBERTa-v3 classification, Monte Carlo Dropout uncertainty quantification, and temperature-scaled calibration for response-level hallucination detection is presented.

Varun Teja Chundru, Debasmita Biswas · 0 citations
#artificial intelligence Preprint Sep 2026

Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection

This work investigates whether an LLM can use an alternative, low level, symbolic competence such as SQL for unsupervised hallucination detection in some high level task and finds that this approach improves on direct prediction and competes with state-of-the-art hallucination detection methods, while not requiring dom...

Renato Vukovic, Hsien-Chin Lin, Carel van Niekerk et al. · 0 citations

Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons

This work proposes a five-step diagnostic protocol covering feature correlation, bootstrap stability, sparse versus dense ranking disagreement, intervention baselines, and cross-dataset evaluation as a minimum standard for sparse-neuron localization claims, and investigates prior work using the proposed approach.

Huseyin Cavus, Sebin Sabu, J. Spear et al. · 0 citations

Alleviating Hallucination in Reasoning Tasks with Training-Free Uncertainty-Guided Steering

USteer, a simple, training-free steering mechanism that adjusts a model's layer-wise activations during inference using the gradient of a confidence measure with respect to the activations, demonstrates that confidence signals can be leveraged not only for detection, but also for effective inference-time control of mod...

Li-Tian Liu, Qi-Qi Hou, Yu-Bin Jian et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.