Reinforcement Learning for Ultrasound Image Analysis: A Scoping Review
TL;DR
RL in ultrasound imaging is an emerging field of research and has advantages for sequential workflow optimization tasks, but most approaches are at an early stage and have been tested on small datasets, lack consistent evaluation protocols, and report limited clinical translation.
Abstract
Machine learning using supervised approaches has been widely applied to ultrasound image analysis. In contrast, reinforcement learning (RL), which is well suited for sequential decision-making, is underexplored. Ultrasound workflows involve sequential subtasks like image acquisition, quality assessment, summarization, and interpretation that can be integrated into RL frameworks. This scoping review examined RL applications in ultrasound. A comprehensive search was conducted in Scopus, PubMed, Embase, and MEDLINE for studies published between 2015 and 2026. Eligible studies used RL with clinical ultrasound data, and the data were summarized by application area, methods, and anatomical targets. The review also provides a brief overview of RL concepts as foundational knowledge for understanding various RL formulations used. From 326 records retrieved, 39 studies were included. Most studies used model-free deep RL, with value-based methods being the most common, particularly Deep Q-Network (DQN) and its variants, on retrospective data across diverse anatomical targets, with breast, fetal, and uterine ultrasound being the most frequently represented categories. RL was used to automate tasks including navigation, plane localization, landmark detection, and video summarization. RL in ultrasound imaging is an emerging field of research and has advantages for sequential workflow optimization tasks. Most approaches are at an early stage and have been tested on small datasets, lack consistent evaluation protocols, and report limited clinical translation.