Skip to content
Open access

Generalizable CT vision-language modeling for population health and disease risk

Sep 2026 · npj Digital Medicine · 1 citation

TL;DR

Percival is introduced, a CT-native VLM trained on more than 400,000 CT-report pairs from the Penn Medicine BioBank using a dual-encoder symmetric contrastive framework, and evaluated against alternative foundation-model paradigms, indicating that CT-VLMs uncover latent structure aligned with clinical, physiological, and prognostic variation across the disease-prevalence spectrum.

Abstract

Vision-language foundation models (VLMs) for computed tomography (CT) are emerging tools that learn generalizable representations from large-scale clinical imaging data. While these models can predict task-specific labels, the extent to which their representations capture the clinical, physiological, and longitudinal variation of real-world patient populations remains unclear. We introduce Percival, a CT-native VLM trained on more than 400,000 CT-report pairs from the Penn Medicine BioBank using a dual-encoder symmetric contrastive framework. Across over 20,000 held-out participants, Percival’s latent space aligns with demographic, physiological, and laboratory variation, and supports phenome-wide associations across the electronic health record. We evaluate Percival against alternative foundation-model paradigms, including vision-only contrastive and multi-organ segmentation, demonstrating that vision-language pretraining captures clinical information not fully accessible to vision-only alternatives across disease classification and longitudinal risk modeling. Together, these findings indicate that CT-VLMs uncover latent structure aligned with clinical, physiological, and prognostic variation across the disease-prevalence spectrum.

Read PDF

Similar papers

Review Open access Oct 2026

Vision–Language Models in Ultrasound Imaging: A Comprehensive Review

Ultrasound vision–language models (US-VLMs) must interpret operator-dependent images and dynamic examinations while linking visual evidence to clinical language. This review synthesizes 32 formally published studies identified through a three-database search. We apply an operational three-axis taxonomy: the interaction...

Ze-Yuan Tang, Shu-Kang Zhang, Huan-Jun Wang et al. · 0 citations
Preprint Sep 2026

A visual large language foundational model for medical image recognition using clinician-contributed online resources

Large language models (LLMs) have demonstrated strong capabilities across diverse domains, showing considerable potential in medicine. However, their application in medical settings remains limited by the scarcity of visual question answering (VQA) datasets that capture clinical reasoning and explicit image-text alignm...

Ling-Xuan Hou, Yu-Hua Xie, Yue Hu et al. · 0 citations
Review Open access Sep 2026

From 2D Vision–Language Models to Volumetric Medical AI: Large Language Models and Foundation Models for 3D Medical Imaging

Multimodal large language models (MLLMs) and vision–language models (VLMs) have rapidly entered medicine, demonstrating promising performance in clinical reasoning, radiology report generation, and visual question answering (VQA). However, many current multimodal architectures and pretrained visual backbones remain fun...

Roni Ramon-Gonen, Haya Engelstein · 0 citations
Preprint Sep 2026

Clinical Trajectory Alignment for Medical Vision-Language Pre-training

Medical vision-language pre-training largely follows a visit-level image-report matching paradigm, aligning paired images and reports at individual visits. While effective for static cross-modal correspondence, this paradigm provides limited supervision for longitudinal clinical change, such as whether abnormalities im...

Hui-Min Yan, Xian Yang, Zhi Wang et al. · 0 citations
Open access Aug 2026

Medrecord-CLIP: enhancing fundus disease diagnosis via EHR-guided vision-language pre-training

This work proposes MedRecord-CLIP, a knowledge-enhanced foundation model featuring a diagnosis-guided cross-attention mechanism to adaptively extract and fuse salient patient history with diagnostic representations that highlights the critical value of integrating personalized clinical context to enhance the generaliza...

Lei Shi, Wenbin Zhai, Lei Yu et al. · 0 citations
#federated learning Review Open access Oct 2026

Foundation Models and Vision-Language Models in Medical Imaging: A Review

Foundation models (FMs) and vision-language models (VLMs) have emerged as a transformative paradigm in artificial intelligence, reshaping the landscape of medical image analysis and clinical decision support. Building upon large-scale pretraining on heterogeneous datasets, these models exhibit unprecedented generalizat...

Saeed Mohsen, Leena Mahmoud Lulu, Mohamed Eassa et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.