Can ChatGPT Guide Effective Thoracic Imaging Based on ACR Appropriateness Criteria?
TL;DR
ChatGPT demonstrated substantial and moderate concordance with ACR recommendations for basic thoracic imaging modalities, particularly CXR, NECT and CECT, but showed variable consistency for advanced imaging techniques, suggesting that ChatGPT may have a limited role as a supportive decision-making tool for non-radiologist clinicians.
Abstract
Aim: This study aimed to assess the concordance between imaging modality recommendations generated by ChatGPT and the American College of Radiology (ACR) Appropriateness Criteria for thoracic clinical scenarios.Materials and Methods:68 detailed clinical case scenarios representing 16 thoracic diagnostic categories defined in the ACR Appropriateness Criteria were presented to ChatGPT in a simulated clinical decision making format. For each scenario, imaging modalities suggested by ChatGPT were classified as “usually appropriate,” “may be appropriate,” or “usually not appropriate.” Categorical differences were analyzed using the chi-square test. Agreement between ChatGPT and ACR ratings was evaluated using Cohen’s kappa coefficient, and the McNemar test was applied to compare appropriate versus inappropriate classifications.Results: ChatGPT most frequently recommended non-contrast-enhanced computed tomography(NECT) (60.29%), chest radiography (CXR) (57.35%), and contrast-enhanced computed tomography (CECT) (54.41%). Other modalities, including chest ultrasound(USG), non-contrast-enhanced thoracic magnetic resonance imaging (NEMRI),contrast-enhanced thoracic magnetic resonance imaging(CEMRI), and positron emission tomography (PET), were less frequently recommended. Cohen’s kappa analysis showed substantial agreement for CXR (κ = 0.625), moderate agreement for NECT (κ = 0.460) ,CECT (κ = 0.520) , and fair agreement for the other modalities. McNemar’s test revealed no significant differences between ChatGPT and ACR classifications for any modality except USG . Conclusion: ChatGPT demonstrated substantial and moderate concordance with ACR recommendations for basic thoracic imaging modalities, particularly CXR, NECT and CECT, but showed variable consistency for advanced imaging techniques. These findings suggest that ChatGPT may have a limited role as a supportive decision-making tool for non-radiologist clinicians.Keywords: Artificial Intelligence, Appropriateness Criteria, Radiology, Diagnostic Imaging