Skip to content
Open access

A framework for extraction of clinical information from radiological mammography reports using large language models and retrieval augmented generation

Sep 2026 · BMC Medical Informatics and Decision Making · Vol 26 · 1 citation · ⚡ 1 influential · 37 references
Medicine

TL;DR

A structured Retrieval-Augmented Generation (RAG) framework that leverages annotated mammography reports as in-context demonstrations, enabling joint named entity and relation extraction without task-specific fine-tuning is proposed and achieves performance comparable to fine-tuned BETO models in low-resource settings, while significantly reducing training and deployment overhead.

Abstract

Extracting structured information from free-text radiology reports is essential for downstream clinical analysis and decision support. In mammography, this challenge is amplified by heterogeneous writing styles, the absence of standardized terminology across institutions, and limited availability of annotated Spanish datasets. Large language models (LLMs) offer a compelling alternative by enabling few-shot generalization without task-specific fine-tuning. We propose a structured Retrieval-Augmented Generation (RAG) framework that leverages annotated mammography reports as in-context demonstrations, enabling joint named entity and relation extraction without task-specific fine-tuning. The framework is evaluated on Spanish BI-RADS–style mammography reports and achieves performance comparable to fine-tuned BETO models in low-resource settings, while significantly reducing training and deployment overhead. In NER, GPT-based few-shot models achieved competitive performance (0.89–0.94 F\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_1$$\end{document}) relative to the fine-tuned BETO baseline (0.97 F\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_1$$\end{document}), despite requiring no task-specific training. In RE, LLMs showed moderate performance (up to 0.78 F\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_1$$\end{document}), remaining below the supervised BETO model (0.99 F\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_1$$\end{document}), reflecting the greater sensitivity of RE to boundary errors and cross-sentence context. Inference costs were low (fractions of a cent per report) and latency remained within seconds, enabling practical deployment scenarios. Local open-weight models preserved privacy and runtime efficiency but exhibited substantially lower accuracy. Few-shot RAG combined with modern LLMs provides a viable, data-efficient alternative for structuring Spanish mammography reports, particularly in low-resource or rapid-deployment settings. While fine-tuned encoders remain preferable for high-accuracy RE, the proposed framework offers a practical balance between performance, operational cost, and accessibility. A remaining limitation is the need for an initial annotated subset to populate the RAG store; future work will explore weak supervision, multimodal extensions, and cross-site generalization.

Read PDF

Similar papers

Comparison of techniques for fine-tuning open-weight models for entity extraction from radiology reports

For narrow, high-value clinical label-extraction tasks, distilling real reports, rather than generating synthetic ones, is what closes the gap to a hosted model, enabling a private, low-cost, version-stable on-premises alternative.

Aawez Mansuri, Kush Mehta, Mohammadreza Chavoshi et al. · 0 citations

Representation-guided in-context learning for medical image interpretation with multimodal large language models

RG-ICL is introduced, a training-free inference framework that retrieves query-aligned demonstrations using frozen encoders, without task-specific parameter updates, and indicates that for medical image interpretation, curating which reference cases an MLLM sees is a practical alternative to retraining it.

Min-Da Zhao, Fang-Yu Hu, Yan Luo et al. · 0 citations
#natural language process... Preprint Sep 2026

SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer

SIFTING enables accurate, structured, and traceable information extraction from clinical free-text documents and ensures data control, reproducibility, and verifiable outputs that can support clinical validation and workflow integration.

Mirco Hess, Gerben van Veenendaal, J. Wakkie et al. · 0 citations

Enhancing Image-Text Alignment in Chest X-ray Datasets by Reducing External References via a Fine-Tuned Large Language Model Pipeline.

An automated, data quality-oriented preprocessing pipeline that rewrites radiology reports at the sentence level to reduce external references is developed and can serve as an effective preprocessing step for improving the suitability of chest X-ray report datasets for multimodal model development.

Yu-Ruei Chen, Chang-Fu Kuo, Ching-Heng Lin · 0 citations
Open access Sep 2026

Cross-Lingual Transfer for Mammography Report Classification in Low-Resource Settings

Spanish initialization was compatible with subsequent low-resource Russian adaptation in this experiment, while the target-language lexical model remained stronger, limited to the two datasets, grouped split, XLM-R configuration, and small, imbalanced target-data regime evaluated.

Anuar Dosmaganbetov, T. Zhaksylyk, B. Abdikenov · 0 citations
#artificial intelligence Preprint Aug 2026

Error Detection for PET/CT Radiology Reports: Domain-Specific vs Large Language Models

It is suggested that domain-specific training matters more than model scale for PET/CT report error detection, supporting compact models as an accurate and computationally efficient approach to automated radiology report quality assurance.

Hermione Warr, Harry Anthony, Lilli J. Freischem et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.