Real-World Effects of a Sepsis Early Detection Model Integrated into Clinical Workflow: A Quasi-Experimental Study
TL;DR
Integration of ESM into clinical workflow was associated with improved patient outcomes, reduced antibiotic use, and decreased harm from overtreatment, without evidence of increased harm from delayed treatment, supporting a positive net clinical benefit and the safety and effectiveness of ESM under Software as a Medical Device principles.
Abstract
Abstract Background Sepsis is a life-threatening condition in which delayed recognition and treatment are associated with increased mortality. While predictive models such as Epic’s Early Detection of Sepsis Model (ESM) were developed to support early intervention, their real-world effects after integration into clinical workflows remain difficult to evaluate. Objectives This study aimed to evaluate the real-world effects of ESM integrated into clinical workflow on clinical outcomes, antibiotic use, and harm–benefit trade-offs. Methods We conducted a quasi-experimental study in a single health care system using encounter-level data from inpatient settings. Inpatient mortality, prolonged hospitalization, antibiotic use, and sepsis prevalence (defined by electronic health record [EHR]) were compared between the preacquisition period (June 3, 2023–June 24, 2024) and the online period (August 21, 2024–December 26, 2024) when the model became visible to clinicians. We also applied a counterfactual framework using models trained on preacquisition data to estimate expected outcomes without ESM and to quantify harms related to overtreatment and delayed treatment. Results Among 89,469 encounters, 75,215 occurred during the preacquisition period and 14,254 during the online period. In unadjusted analyses, inpatient mortality, prolonged hospitalization, antibiotic use, and EHR-defined sepsis proportion all decreased during the online period (all p ≤ 0.006). In the counterfactual analyses, observed outcomes were lower than expected without ESM for mortality (1.21 vs. 1.94%; p < 0.001), prolonged hospitalization (5.56 vs. 8.14%; p < 0.001), and antibiotic use (43.52 vs. 47.74%; p < 0.001). False positive harm (37.72 vs. 42.31%; p < 0.001) was also lower than expected. Conclusion Integration of ESM into clinical workflow was associated with improved patient outcomes, reduced antibiotic use, and decreased harm from overtreatment, without evidence of increased harm from delayed treatment, supporting a positive net clinical benefit and the safety and effectiveness of ESM under Software as a Medical Device principles.