Skip to content

InfiMed2: A Generalist Medical Multimodal Foundation Model from Contextual Evidence and Stability-Aware Supervision

Sep 2026 · arXiv.org · Vol abs/2609.34798 · 0 citations · 30 references
Computer Science

TL;DR

This work introduces InfiMed2, a family of 4B and 27B generalist medical multimodal foundation models built around stage-aware data design, and curates a 55.68B-token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing.

Abstract

Recent medical multimodal models have benefited from larger corpora, broader modality coverage, and stronger reasoning-oriented training, yet effective data design across continued pretraining (CPT) and post-training remains challenging. Medical sources vary substantially in structure, granularity, and information density, and their utility shifts as training progresses from broad knowledge acquisition to late-stage consolidation. Meanwhile, post-training is often dominated by short-form visual question answering, providing limited supervision for informative and answer-consistent explanations. We introduce InfiMed2, a family of 4B and 27B generalist medical multimodal foundation models built around stage-aware data design. We curate a 55.68B-token corpus that combines broad clinical knowledge with context-rich biomedical visual evidence through source-specific processing. Our CPT pipeline first adapts the vision encoder, then builds broad medical knowledge, and finally transitions to an evidence-focused data mixture during learning-rate decay. For supervised fine-tuning (SFT), we regenerate visual question-answering responses using answer stability, answer-masked reconstruction, and correctness-constrained selection to produce more informative and answer-consistent supervision. The 4B model is further optimized with reinforcement learning with verifiable rewards (RLVR). Across five medical multimodal benchmarks, InfiMed2-4B achieves 66.73% mean accuracy after RLVR, surpassing the larger Qwen3.5-9B, while InfiMed2-27B reaches 73.72%, the highest among the evaluated open-weight models.

View source

Similar papers

Sep 2026

Lingshu: Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning.

This work proposes a comprehensive data-curation procedure that efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general domain data and develops a medical-specialized MLLM, Lingshu-Med, which consistently outperforms existing multimodal baselines...

Wei-Wen Xu, Hou-Pong Chan, Long Li et al. · 0 citations
Sep 2026

MedCure: Medical Data Curation for Efficient Vision–Language Pretraining

Medical foundation models (FMs) typically rely on contrastive vision-language pretraining over large-scale datasets, yet such datasets often exhibit substantial heterogeneity in data quality and demand extensive computational resources. Recent studies suggest that data quality matters more than data quantity, but how t...

Chong Wang, Feng-Bei Liu, Yu-Yuan Liu et al. · 0 citations
#artificial intelligence Preprint Aug 2026

Dense Clinical Contrasts Enhance Medical Knowledge Updating in Large Language Models

Results show that medical knowledge updating depends not only on the update algorithm, but also on how knowledge is structured as supervision, and suggest that EMQ gives the most stable external transfer and retention among same-budget SFT variants.

Yangmin Huang, Shu Quan, He Geng et al. · 0 citations

Representation-guided in-context learning for medical image interpretation with multimodal large language models

RG-ICL is introduced, a training-free inference framework that retrieves query-aligned demonstrations using frozen encoders, without task-specific parameter updates, and indicates that for medical image interpretation, curating which reference cases an MLLM sees is a practical alternative to retraining it.

Min-Da Zhao, Fang-Yu Hu, Yan Luo et al. · 0 citations
#machine learning Preprint Sep 2026

SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

A unified learning paradigm is proposed that jointly optimizes predictive performance and modality-specific explainability by identifying the most informative elements inside each modality that contribute to diagnostic decisions by employing a matrix-based Renyi's $\alpha$-order entropy functional under the assumption...

Yu-Qing Yang, Alexander Schmatz, Zhao-Zhao Ma et al. · 0 citations
Preprint Aug 2026

MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis

This work introduces MedReaMM, a benchmark specifically designed to evaluate models'ability to synthesize heterogeneous clinical evidence consisting of detailed patient histories alongside multiple medical images into accurate differential diagnoses under a complete-information paradigm.

Lai Wei, Yu-Chao Chen, Zhenbiao Cao et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Jun 3, 2026

MIT researchers teach AI models to interpret charts

The new ChartNet training dataset could improve the accuracy of vision-language models that help analyze business trends or interpret scientific figures.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.