Multi-agent planning and generation of radiology reports using multimodal agentic frameworks

Abstract

Radiology reports follow a prescribed structure, such as they describe the examination type, patient history, comparisons to prior studies, technical parameters, detailed findings and a concluding impression. Automating such reporting requires more than simple text generation, with models that must adhere to this structure and accurately interpret medical images and clinical context. Here we propose a modular, multi‑agent system for radiology report generation that organizes the task into planning, generation and critique phases. The system begins with a planner agent that analyses the imaging modality, clinical indication and institutional reporting guidelines to produce a plan listing the expected sections and sub‑sections of the report. Specialized section agents then generate content for each portion: one drafts the exam description and technique, another summarizes the patient history and prior comparisons, a findings agent enumerates observations by organ system, and an impression agent synthesizes the salient findings into a diagnostic conclusion. Finally, a critique agent, which is multimodal capable of interpreting both text and imaging, reviews the assembled report for factual accuracy, completeness and conformity to the planned structure, and provides feedback for iterative refinement. By separating planning, generation and critique into distinct agents, the system is adaptable to new imaging modalities or institutional templates, and can be enhanced with domain-specific components such as knowledge graph retrieval or rule-based checkers.