Skip to content
Open access

An explainable biomedical foundation model via large-scale concept-enhanced vision-language pretraining.

Aug 2026 · Nature Biomedical Engineering · 0 citations · 43 references
Medicine

TL;DR

Across a large-scale benchmark covering 78 datasets in 10 imaging modalities, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations, and represents a critical milestone towards the widespread clinical adoption of AI.

Abstract

Artificial intelligence for medical imaging is required to be accurate and interpretable to clinicians. However, current multimodal biomedical foundation models often prioritize performance over explainability. Here we present ConceptCLIP, an explainable biomedical foundation model that achieves state-of-the-art diagnostic accuracy while delivering human-interpretable explanations across diverse imaging modalities. We curate MedConcept-23M, a large-scale dataset comprising 23 million biomedical image-text-concept triplets. Leveraging this dataset, we pretrain ConceptCLIP via joint image-text and region-concept alignment for precise and interpretable medical image analysis. Across a large-scale benchmark covering 78 datasets in 10 imaging modalities, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations. In a clinician user study spanning three modalities, the concept-based explanations provided by ConceptCLIP help clinicians verify model predictions and identify potential errors. As an explainable biomedical foundation model, ConceptCLIP represents a critical milestone towards the widespread clinical adoption of AI, thereby advancing trustworthy AI in medicine.

Read PDF

Similar papers

Preprint Sep 2026

A visual large language foundational model for medical image recognition using clinician-contributed online resources

Large language models (LLMs) have demonstrated strong capabilities across diverse domains, showing considerable potential in medicine. However, their application in medical settings remains limited by the scarcity of visual question answering (VQA) datasets that capture clinical reasoning and explicit image-text alignm...

Ling-Xuan Hou, Yu-Hua Xie, Yue Hu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

A visual large language foundational model for medical image recognition using clinician-oriented social media

A FOundational LLM Trained on ThoughtMed-1M (FOLTMed), a scalable paradigm for advancing research on clinically grounded multimodal LLMs, achieved state-of-the-art performance across 42 medical VQA benchmark datasets, with a macro accuracy of 85.4%, and generated more clinically coherent responses on the ThoughtMed-1M...

Ling-Xuan Hou, Yu-Hua Xie, Yue Hu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Concept-Grounded Reasoning with Prompt-Driven Localization for Interpretable Structured Report Generation

Experiments on BUS-CoT and IU X-ray datasets demonstrate consistent improvements in diagnostic accuracy, concept consistency, and report quality over strong general-purpose and medical MLLMs, indicating that concept-grounded reasoning better aligns generation with clinical decision processes.

Xin-Yue Xu, Hong-Bin Lin, Juan-Gui Xu et al. · 0 citations
Sep 2026

MedCure: Medical Data Curation for Efficient Vision–Language Pretraining

Medical foundation models (FMs) typically rely on contrastive vision-language pretraining over large-scale datasets, yet such datasets often exhibit substantial heterogeneity in data quality and demand extensive computational resources. Recent studies suggest that data quality matters more than data quantity, but how t...

Chong Wang, Feng-Bei Liu, Yu-Yuan Liu et al. · 0 citations
Conference Open access Sep 2026

Hierarchical Conditional Energy Modeling for Medical Vision–Language Pretraining

HCE-CLIP is proposed, a vision–language pretraining framework that formulates medical image–text alignment as a hierarchical label-conditional energy modeling problem and introduces a hierarchical contradiction-based metric that quantifies logical inconsistencies between fine-grained disease predictions and higher-leve...

Cheng-Sheng Mao, Yuan Luo · 0 citations
#computer vision Sep 2026

InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision

This work introduces InfiMed2, a family of 4B and 27B generalist medical multimodal foundation models built around stage-aware data design, and curates a 55.68B-token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing.

Guang-Hao Zhu, Ze-Yu Liu, Zhitian Hou et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.