Skip to content
Preprint

Efficient Multi-Granularity Knowledge Transfer for Radiology Report Generation

Oct 2026 · 0 citations · 51 references
Computer Science

TL;DR

A novel and compact Efficient Multi-Granularity Knowledge Transfer (\textbf{EMGKT}) method to address the above issues and develops a mixture of disease diagnosis expert classifiers to enhance knowledge extraction.

Abstract

Radiology report generation can automatically generate clinical descriptions from X-ray images, thereby significantly improving the efficiency of radiologists. This task is challenging because it requires medical knowledge to accurately identify diseases and describe them in a professional manner. However, existing methods often overlook the importance of enhancing medical knowledge in describing pivotal areas, a capability that requires models to effectively extract and aggregate knowledge at multiple levels of granularity. Accordingly, we herein propose a novel and compact Efficient Multi-Granularity Knowledge Transfer (\textbf{EMGKT}) method to address the above issues. First, we encode global knowledge embeddings using a medical vision-language model, which provides contextual medical knowledge. Moreover, we devise a novel Fine-Grained Knowledge Distillation (FGKD) training task which efficiently extract fine-grained knowledge. Specifically, the FGKD training task contains teacher embeddings and student embeddings. Teacher embeddings are encoded using extra priors; while student embeddings are learned from the teacher embeddings through knowledge distillation. During inference, the student embeddings are used to enhance fine-grained knowledge while the teacher embeddings are discarded, resulting in negligible computational costs and no need for extra priors. Finally, we further develop a mixture of disease diagnosis expert classifiers to enhance knowledge extraction. The classifiers are initialized using disease embeddings and are modeled as different experts to address various granularity features. Notably, \textbf{EMGKT} can be efficiently applied to most existing methods. Extensive experiments are conducted on two widely-used public datasets and various baselines, which demonstrates the effectiveness and transferability of \textbf{EMGKT}.

View source

Similar papers

Preprint Nov 2026

Findings-aware Language-Image Pretraining for CT Volume Interpretation

Computed tomography (CT) provides diverse patient information and various clinical tasks, for which numerous deep learning models have been investigated. Recent years have seen the development of vision-language foundation models that can comprehensively extract clinical features, driven by representation learning that...

Takuro Shimaya, Junya Inoh, M. Saiko · 0 citations

Representation-guided in-context learning for medical image interpretation with multimodal large language models

RG-ICL is introduced, a training-free inference framework that retrieves query-aligned demonstrations using frozen encoders, without task-specific parameter updates, and indicates that for medical image interpretation, curating which reference cases an MLLM sees is a practical alternative to retraining it.

Min-Da Zhao, Fang-Yu Hu, Yan Luo et al. · 0 citations
Sep 2026

Concept-Enhanced Multi-Scale Cross-Modal Alignment for Medical Visual Representation Learning.

A Concept Clause Decomposition method is designed to extract semantically complete descriptions of pathological findings or radiology manifestations from medical reports as medical concept clauses, which are then utilized within a multi-granularity cross-modal alignment framework to enhance medical concept perception a...

Xiang-Min Kong, Xi-Bin Jia, Da-Wei Yang et al. · 0 citations
Sep 2026

Comorbidity-Aware Radiology Report Generation.

Chest X-ray report generation systems are valuable for assisting disease diagnosis and improving healthcare efficiency. However, existing methods still face two key challenges. First, multiple diseases often co-occur, leading to a combinatorial explosion of label combinations and sparse supervision for learning a gener...

Hong-Ze Zhu, Hong Liu, Ya-Wen Huang et al. · 0 citations
#machine learning Preprint Oct 2026

Homogeneous Semantic Alignment and Hierarchical Expert Routing for Radiology Report Generation

Radiology report generation (RRG) aims to convert medical images into diagnostic texts to assist in clinical decision-making and alleviate the workload of physicians. Although existing methods have made extensive progress in cross-modal interaction and the incorporation of external priors, the distribution shift of und...

Er-Jian Zhang, Jia-Yuan Ma, Lie-Jun Wang et al. · 0 citations
#machine learning Preprint Oct 2026

Diagnostic Loss: Asymmetric Supervision for Radiology Report Generation

Automated radiology report generation requires models to do more than produce fluent clinical narratives: they must explicitly state supported diagnoses while avoiding unsupported ones. However, existing medical vision-language models (MedVLMs) are typically trained with token-level maximum likelihood objectives, which...

Ignacy Kolton, Wen-Xuan Li, Zong-Wei Zhou et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.