RadLens: Radiology Report Generation Using a Vision Transformer and Retrieval-Augmented Generation

Abstract

Automated radiology report generation systems usually produce single-section free-text reports and often lack radiologist review workflows, limiting their clinical use. This study proposed a RadLens system (Radiology Report Generation Using a Vision Transformer and Retrieval-Augmented Generation), a five-stage AI-assisted pipeline for generating structured draft reports from chest X-ray images as a case study. Each report contains five fields: clinical summary, key findings, red flags, follow-up reminders, and missing information. Three pre-trained vision transformer encoders, Bidirectional Encoder representation from Image Transformers (BEiT), Swin Transformer, and Vision Transformer (ViT), were fine-tuned on the public Indiana University Chest X-Ray dataset using contrastive loss. The fine-tuned encoder features were then used with Claude Sonnet 4.6 via Amazon Bedrock to generate structured reports. Claude Sonnet 4.6 was used with prompt engineering and Retrieval-Augmented Generation (RAG). For each test image, RAG retrieved the top three visually similar training cases using cosine similarity. RAG improved BLEU-1 scores from 0.1034 to 0.2105 for BEiT (104%), from 0.1130 to 0.2061 for Swin Transformer (82%), and from 0.0955 to 0.2140 for ViT (124%). BERTScore F1 increased from 0.8439, 0.8436, and 0.8435 to 0.8701, 0.8690, and 0.8708, respectively, suggesting improved semantic similarity. RadLens also includes a radiologist review interface for reviewing, editing, approving, or rejecting generated drafts before clinical use.