Skip to content
Log in
Open access

A Metric-Aware Analysis of Trigger-Guided Adapter Training for Hallucination Mitigation

Junjun Zhang Giseop Noh
2026 · IEEE Access · Vol 14, pp. 113956-113970 · 0 citations · 26 references

TL;DR

TruthShield is presented, a metric-aware trigger-guided QLoRA adapter training and evaluation pipeline for hallucination-aware language model adaptation and suggests that trigger-guided adapter training may learn surface-level response patterns without clear evidence of semantic hallucination mitigation under the current single-judge setting.

Abstract

Large language models can produce fluent responses that are unsupported by the provided evidence or inconsistent with reference answers. Adapter-based fine-tuning offers a practical way to modify model behavior without updating all model parameters, but its effect on hallucination mitigation depends strongly on how improvement is measured. This paper presents TruthShield, a metric-aware trigger-guided QLoRA adapter training and evaluation pipeline for hallucination-aware language model adaptation. We construct hallucination-related failure cases from baseline evaluations, assign rule-based failure labels, generate counterexample triggers, and train a Mistral-7B adapter to respond to these trigger patterns. We then compare the baseline model and the adapter across TruthfulQA generation, TruthfulQA multiple choice, and HaluEval using two evaluation views: strict matching and an LLM judge. The adapter substantially improves strict scores across all tasks, increasing strict accuracy by 1.0000 on both TruthfulQA settings and by 0.6957 on HaluEval. However, these gains do not translate into judge-based improvement: judge scores remain nearly unchanged for TruthfulQA generation and decrease for TruthfulQA multiple choice and HaluEval. The results suggest that trigger-guided adapter training may learn surface-level response patterns without clear evidence of semantic hallucination mitigation under our current single-judge setting. This finding highlights the risk of overestimating reliability gains when strict metrics are used without complementary semantic evaluation.

DOI Semantic Scholar
Read PDF

Similar papers

Preprint Aug 2026

Decomposed Entailment for Factuality Checking and Hallucination Detection

HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection, is presented, a lightweight, reference-free, and black-box framework for hallucination detection that is evaluated not only on summarization but across a broader range of source-grounded generation settings.

Achir Oukelmoun, N. Semmar, Gäel de Chalendar · 0 citations

Detecting and Mitigating Hallucinations in Large Language Models: A Comparative Study of Generative and Transformer-Based Approaches

The results suggest that no single architecture guarantees factual reliability, however, contextual grounding and verification mechanisms can significantly improve response quality and highlight the importance of combining language modelling capabilities with grounding strategies to support the development of more reliable AI systems.

Rupinder Kaur, Sangeetha Kaithakkadu, Thankachan · 0 citations
Preprint Jul 2026

SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems

A unified toolkit and interactive web UI for detecting contextual hallucinations in retrieval-augmented, agentic, and memory-grounded LLM systems, and as a faithfulness gate within long-term memory systems is demonstrated.

Julia Belikova, Rauf Parchiev, Mikhail Filimonov et al. · 0 citations
Preprint Jul 2026

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why. To close this gap, we introduce Grounding-Aware Sensitivity by Perturbation (GASP), a span-level detector that scores each answer sentence by how strongly its likelihood depends on the retrieved evidence, a quantity we term grounding sensitivity. GASP holds the answer fixed and re-scores it under the full context, under no context, and with each chunk removed, then measures the log-likelihood drops and Jensen-Shannon divergences (JSD). The likelihood of a grounded sentence collapses once its supporting passage is removed, whereas a hallucinated sentence is almost unaffected, a contrast we interpret by casting decoding as a random nonlinear iterated function system (RNIFS). We evaluate GASP on three benchmarks (RAGTruth, TofuEval, RAGBench) with three instruction-tuned scorers from two model families (Qwen2.5-0.5B, Qwen2.5-1.5B, and SmolLM2-1.7B) under a leakage-clean protocol. On RAGTruth it reaches a response-level area under the ROC curve (AUC) of about 0.73 and a span-level AUC of about 0.67, improving significantly over perplexity and by clear margins over length, whole-context natural language inference (NLI), and self-consistency baselines. The only baseline competitive at the span level is a well-configured chunk-level entailment verifier, which requires a separate model, whereas a training-free threshold on the grounding features matches the trained classifier without labeled data and serves as the default detector. Beyond RAGTruth, the signal transfers to TofuEval but not to short-answer question answering in RAGBench, showing GASP is best suited to outputs constructed from the retrieved context rather than answers recoverable from parametric knowledge.

M. Bouke · 0 citations