Experiments on the SLAKE and MIMIC-CXR datasets demonstrate that CAST consistently outperforms strong baselines and surpasses decoding strategies reliant on ground truth, and indicates that compact, automatically selected regions provide highly effective contrastive guidance without expert annotations.
Abstract
Medical vision-language models (Med-VLMs) have demonstrated strong performance on medical visual question answering, yet they remain prone to hallucination, generating clinically unsupported statements that are insufficiently grounded in image evidence. Mitigation methods applied during decoding offer a practical solution, but they typically lack anatomical awareness or rely heavily on ground truth annotations, which limits their applicability. We propose Counterfactual Anatomy-guided Spatial-Temporal decoding (CAST), a framework that operates entirely during inference and requires no manual annotations for anatomically grounded hallucination mitigation. CAST automatically discovers anatomical regions relevant to the given query through broad medical segmentation. It then selects a compact, causally informative area using counterfactual intervention based on the drop in answer likelihood under occlusion. Guided by this chosen region, CAST performs a unified contrastive decoding process, combining classifier-free guidance to correct spatial attention with stepwise temporal contrast to regulate generation dynamics. Experiments on the SLAKE and MIMIC-CXR datasets across three Med-VLMs demonstrate that CAST consistently outperforms strong baselines and surpasses decoding strategies reliant on ground truth. Our results indicate that compact, automatically selected regions provide highly effective contrastive guidance without expert annotations, offering a practical and generalizable solution for improving spatial grounding and reducing hallucinations. Code is available at https://github.com/csyifan/CAST.
Verifier-Guided Decoding (VGD), a decoding framework in which a lightweight verifier examines each emerging object mention, rolls back the KV cache when the mention is identified as high risk, suppresses the object and its synonyms, and regenerates the affected continuation, achieves state-of-the-art object hallucination reduction.
Through an exhaustive evaluation of over 20 state-of-the-art MLLMs, HoloCount reveals a critical performance gap: even top-tier models degrade significantly as tasks transition from perception to complex analytical reasoning and adverse scenarios.
This work proposes CounterfactualLVLM, a training-free and plug-and-play framework that mitigates object hallucinations via small-model-assisted counterfactual reasoning and highlights the power of counterfactual guidance as a simple yet effective paradigm for enhancing factual grounding in LVLM-based multi-modal reasoning.
Xilin Li, Boyue Wang, Xiaoqian Ju et al.· Multimedia Systems· 0 citations
OBJECTIVE
Most hallucination mitigation for large language models (LLMs) operates post-hoc, leaving safety-critical clinical deployment without real-time warning capability. We present a calibrated hidden-state probing pipeline that enables token-time hallucination detection under explicit false-positive-rate (FPR) constraints, making it suitable for streaming clinical deployment.
METHODS
We formulate token-time hallucination detection as a constrained sequential decision problem separating non-circular supervision construction (using ROUGE-L, exact-choice, and exploratory NLI verifiers), FPR-constrained operating-point selection, and downstream intervention evaluation into a modular, reusable protocol. A lightweight two-layer hidden-state probe classifies each generated token into one of three risk states (Safe, AtRisk, or Hallucinating), where the intermediate AtRisk state captures pre-error instability. We validate across four medical QA benchmarks (Endoscopy, PubMedQA, MedHallu, MedQA-USMLE) spanning approximately 44,500 generation trajectories and three backbone LLMs (Qwen3-8B, Llama-3.1-8B-Instruct, BioMistral-7B).
RESULTS
On the public biomedical QA benchmarks the probe yields viable detection across all backbone-dataset combinations: EDR@5 up to 0.390 (PubMedQA) and 0.352 (MedHallu) at the strict cap γ≤0.10, rising to 0.587 with 29%-59% hallucination reduction at the monitoring cap γ≤0.20. The trigger fires roughly 10-35 tokens before error onset, whereas a post-hoc check has zero lead time by construction. A single-institution Endoscopy corpus serves as a case study (EDR@5 0.613, Llama-3.1), treated as illustrative because its small confirmed-correct denominator (26-52 per backbone) yields wide confidence intervals. Probe latency is ∼10ms per answer versus ∼10s for multi-sample baselines.
CONCLUSION
Hidden-state probing with FPR-constrained calibration provides a practical, low-latency solution for real-time hallucination monitoring in clinical LLM deployments. The modular pipeline separating supervision construction, probe training, and trigger calibration is directly reusable with alternative detectors or verifiers.
Zhengqiu Yu, Xiangrong Liu· Journal of Biomedical Inform...· 0 citations
In radiology, multimodal vision-language models (VLMs) are increasingly used for clinical tasks such as report generation and visual question answering. Their adoption has raised important concerns regarding the transparency and trust-worthiness of the generated text reports, as the reasoning process leading to a given report remains opaque for clinicians. In this work, we investigate how post-hoc attribution methods behave in large generative volumetric VLMs that jointly process 3D scans and clinical text, a setting that remains largely unexplored. We propose to use Post-hoc gradient-based attribution that directly links small changes in the input volume to changes in the model's output probability. The faithfulness and spatial specificity of four attribution methods are subsequently characterized and compared when applied to Med3DVLM, a recent medical VLM for image-text understanding. To enable differentiable attribution at inference time, we apply teacher forcing exclusively during the attribution forward pass so that the stochastic generation path is converted into a differentiable computation graph. The cumulative log-likelihood of the generated response is used as the scalar attribution target. Input-level methods, including Saliency and Integrated Gradients, alongside feature-level techniques such as Grad-CAM and Guided Grad-CAM, are computed on volumetric inputs across diverse clinical question types. Qualitative assessment and a quantitative voxel deletion protocol indicate that input-space gradient methods, especially Integrated Gradients, produce spatially selective and causally faithful relevance maps, whereas feature-level methods like Layer Grad-CAM exhibit significant spatial diffusion and fail to provide discriminative localization in multimodal architectures. These results demonstrate that input-level attributions exhibit a significantly steeper drop in model confidence compared to feature-level methods, confirming they are more faithful and spatially precise, while intermediate-layer maps remain diffuse and non-discriminative.
Youcef Abdelhalim, Arif Ahmed Madani, Saouli Rachida et al.· 2026 6th International Confe...· 0 citations
This work proposes Saliency-Driven Perceptual Realignment (SDPR), a training-free framework that mitigates the degradation of visual awareness throughout inference and is robust against hallucinations due to its holistic alignment of visual awareness across the entire generative trajectory.
Pengxu Chen, Yao Zhu, Guangming Zhu et al.· 0 citations