This work presents MedProb, a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation, and shows the probe can be extended to open-ended generation via a rejection-sampling scoring procedure.
Abstract
Medical visual question answering (Med-VQA) is often assumed to require medical fine-tuning, large models, or complex multi-agent pipelines. We revisit this assumption with \textbf{MedProb}, a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation. Across PATH-VQA, SLAKE, and VQA-RAD, MedProb recovers substantially more answer-relevant signal than prompting and performs stronger than medical VLMs and agentic systems. Probing also reduces the apparent gap between small and large models compared to prompting, suggesting that smaller VLMs contain more recoverable Med-VQA signal than generation-based evaluation reveals. Across 14 matched general-purpose and medical VLM pairs, medical adaptation does not consistently improve this linear decodability. Finally, free-text generation exhibits an answer-position bias of up to 10 percentage points, whereas MedProb also has positional bias, however, it is impacted differently than prompting. Our main results target the multiple-choice/multiclass Med-VQA setting; we additionally show the probe can be extended to open-ended generation via a rejection-sampling scoring procedure.
This work proposes MedVCoT, which incorporates latent visual reasoning into the medical visual question answering (VQA) domain, and utilizes the specialized expertise of MedSAM to train a large vision-language model so that it can autonomously generate consistent and continuous latent visual tokens within Visual Chain-...
Bo Xu, Quan-Hao Zhu, Bo-Lin Zhu et al.· Proceedings of the Thirty-Fi...· 1 citation
A multilingual medical VQA benchmark over eight languages is constructed, organized into four representative scenarios that isolate the core capabilities medical VQA requires, and a training-free scenario-aware representation engineering method is proposed, leveraging LVLMs's superior English medical VQA capability to...
Jing-Bo Wang, Sen-Dong Zhao, Haochun Wang et al.· 0 citations
Across six medical multimodal MCQ datasets, this work separate candidate cues from behavioral evidence through prompt- and image-side audits, modality ablations, and matched repairs that preserve the medical target and answer key, showing that medical image-reasoning claims require route-level evidence.
Ben Wang, Yi-Fan Zhang, Jia-Qing Yu et al.· 0 citations
This review explores the transformative impact of Large Language Models on medical question-answering robots and addresses critical challenges faced by LLM-driven systems, such as limited interpretability, restricted applicability across medical domains, and serious concerns about patient privacy.
Medical question answering (QA) plays a crucial role in clinical decision support, yet robust performance requires models to effectively distinguish relevant evidence from topically similar distractors within retrieved contexts. Existing Vietnamese medical QA benchmarks, however, focus exclusively on zero-shot evaluati...
Nhan Phuoc Thanh Tran, P. Huynh, Trương Quốc Tuấn Trương et al.· International Conference on...· 0 citations
A novel multimodal RAG framework tailored for MedVQA is proposed, which leverages multimodal data, including medical images, reports, and generated captions, to provide more accurate clinical answers, and introduces a training paradigm that uses captions as auxiliary supervision, enhancing cross-modal alignment via con...
Mai A. Shaaban, M. Zarei, Adnan Khan et al.· 0 citations
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.