Sep 2026· IEEE journal of biomedical and health informatics· Vol PP· 0 citations
Medicine
TL;DR
A Concept Clause Decomposition method is designed to extract semantically complete descriptions of pathological findings or radiology manifestations from medical reports as medical concept clauses, which are then utilized within a multi-granularity cross-modal alignment framework to enhance medical concept perception and fine-grained medical vision-language alignment.
Abstract
Medical Vision-Language Pre-training (Med VLP) on paired medical images and reports has emerged as a promising direction for learning visual representations. However, current alignment approaches remain insufficient for learning fine-grained pathological details, largely due to the inherent difficulty of tokenizing reports without losing accurate and complete radiological concepts. To ad dress this, we propose a novel Concept-Enhanced Medical Vision-Language Alignment (CoMA) framework, which facilitates the acquisition of medical concepts (e.g., pathological findings) by effectively leveraging the radiologists' experience embedded in the text. Specifically, based on radiologists' cognitive framework, we design a Concept Clause Decomposition method to extract semantically complete descriptions of pathological findings or radiology manifestations from medical reports as medical concept clauses. These clauses are then utilized within a multi-granularity cross-modal alignment framework to enhance medical concept perception and fine-grained medical vision-language alignment. Beyond Global Instance-level Alignment (GIA), we introduce Concept-level Token-wise Alignment (CTA) via a bidirectional cross-attention mechanism, which enhances semantic correspondence at the pathological region level by aligning image patches with concept clauses. At a higher semantic level, we establish a bi-level proto type alignment mechanism, comprising Instance-level Prototype Alignment (IPA) and Concept-level Prototype Alignment(CPA).Through cross-modal prototype clustering, this mechanism pulls together unpaired high-level global features and low-level concept features across image and text, respectively. Extensive experiments across seven super vised downstream dataset-task settings demonstrate that CoMA achieves consistent and competitive performance, particularly in limited-annotation settings.
This work introduces MedPlex (Medical Plexus of Vision and Language), an end-to-end VLM framework that makes text guidance a continuous, clinically grounded component of segmentation learning, and introduces Bi-Fusion (Bidirectional Fusion), visual and textual representations evolve jointly across the encoding hierarch...
Rafi Ibn Sultan, Hui Zhu, Chengyin Li et al.· 0 citations
A unified framework for automatic report generation from SPECT bone scintigrams that integrates domain-adaptive representation learning, fine-grained image–text alignment, and anatomy-guided supervision is proposed, offering a valuable pathway to achieving trustworthy and intelligent diagnostic support within nuclear m...
Tao Song, Qiang Lin, Tong-Tong Li et al.· Applied intelligence (Boston...· 0 citations
HCE-CLIP is proposed, a vision–language pretraining framework that formulates medical image–text alignment as a hierarchical label-conditional energy modeling problem and introduces a hierarchical contradiction-based metric that quantifies logical inconsistencies between fine-grained disease predictions and higher-leve...
Cheng-Sheng Mao, Yuan Luo· Proceedings of the Thirty-Fi...· 0 citations
This work evaluates three independent forms of structured supervision: topological priors via graph self-supervision, dense pixel-level constraints via segmentation, and cross-modal semantic grounding via image-text pairs, and finds that image-text alignment achieves the most superior performance.
He-Xiang Bai, Han-Yang Xu, Xiao-Xue Li et al.· 0 citations
A framework that strengthens clinical alignment by separating and enhancing training-time alignment and inference-time alignment is proposed and experiments show that complementary visual representations with a multi-encoder design and concept-level auxiliary learning help preserve clinically meaningful information.
Yunseo Lee, Hyun Jun Kim, Heeseung Shin et al.· 0 citations
A novel multimodal RAG framework tailored for MedVQA is proposed, which leverages multimodal data, including medical images, reports, and generated captions, to provide more accurate clinical answers, and introduces a training paradigm that uses captions as auxiliary supervision, enhancing cross-modal alignment via con...
Mai A. Shaaban, M. Zarei, Adnan Khan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.