Evaluating an AI-assisted triage workflow for retinal diseases
TL;DR
An AI-assisted negative-screening workflow for diabetic retinopathy, retinal vein occlusion, and age-related macular degeneration using 6,904 color fundus photographs independently graded by three retina specialists supports disease-specific implementation strategies that balance workload reduction against missed positive cases.
Abstract
The clinical value of artificial intelligence (AI) in fundus photography depends on workflow efficiency as well as diagnostic performance. We evaluated an AI-assisted negative-screening workflow for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) using 6,904 color fundus photographs independently graded by three retina specialists. In this workflow, AI-negative images were classified as negative without ophthalmologist review, whereas AI-positive images were referred for human interpretation. For each reader–disease pair, the reference standard was agreement between the other two readers; discordant cases were excluded (DR, 1.2%–2.3%; RVO, 0.3%–0.6%; AMD, 6.1%–11.6%). The workflow reduced direct ophthalmologist review to 7.3%–8.0% of images for DR, 11.7%–11.9% for RVO, and 13.5%–16.8% for AMD. For DR, sensitivity decreased significantly (0.9168 to 0.8838; difference, -0.0330; 95% confidence interval [CI], -0.0495 to -0.0165; p < 0.001), whereas specificity did not change significantly (0.9961 to 0.9983; p = 0.183). For RVO, sensitivity was unchanged (0.9807) and specificity did not change significantly (0.9985 to 0.9988; p = 0.320). For AMD, sensitivity decreased significantly (0.8570 to 0.8445; difference, -0.0124; 95% CI, -0.0215 to -0.0033; p = 0.008), whereas specificity did not change significantly (0.9685 to 0.9843; p = 0.324). In an exploratory image-level analysis for at least one target disease, review decreased to 27.7%–30.1%; sensitivity changed from 0.9122 to 0.9039 (difference, -0.0083; 95% CI, -0.0131 to -0.0035; p < 0.001) and specificity from 0.9659 to 0.9804 (p = 0.334). AI-assisted negative-screening can reduce ophthalmologist workload across multiple retinal diseases, but with disease-specific safety trade-offs: because AI-negative images are not reviewed, reduced sensitivity for DR and AMD means a small proportion of true-positive cases would go undetected, potentially delaying diagnosis and treatment for sight-threatening disease. These results support disease-specific implementation strategies that balance workload reduction against missed positive cases.