Machine learning–based prediction of incident delirium after a 24-hour landmark in critically ill patients with acute pancreatitis: Model development and external validation using the MIMIC-IV and eICU databases
Abstract
Objective To develop, internally validate, and preliminarily externally validate a model predicting incident delirium after the first 24 hours of intensive care in patients with acute pancreatitis (AP). Methods This retrospective prediction study used MIMIC-IV 3.1 for development and eICU-CRD 2.0 for external validation. Adults with AP in their first AP-associated ICU stay were screened consecutively. At ICU hour 24, predictors were restricted to the preceding 24 hours; patients with positive CAM-ICU delirium during this interval were excluded, and the outcome was subsequent strictly assessable positivity. Imputation, feature selection, and tuning were repeated within nested cross-validation. Penalized logistic regression was compared with four nonlinear algorithms. The locked pipeline was applied to eICU without refitting, and SHAP described predictions. Results The development cohort included 446 patients; 164 (36.8%) developed incident delirium. The final model retained minimum Richmond Agitation–Sedation Scale (RASS), Sequential Organ Failure Assessment (SOFA), blood urea nitrogen, and mean heart rate. Internally validated penalized logistic regression yielded an area under the receiver operating characteristic curve (AUROC) of 0.818 (95% confidence interval [CI], 0.777–0.857), an area under the precision–recall curve of 0.731, and a Brier score of 0.163. Random forest performed similarly (AUROC, 0.821; P=0.693). In the strict eICU cohort (46 patients; 10 events; 23 hospitals), the locked model yielded an AUROC of 0.839 (95% CI, 0.704–0.944) and a Brier score of 0.145. SHAP ranked minimum RASS and SOFA as the leading contributors. Conclusions A parsimonious landmark model may help identify AP patients at risk of subsequent CAM-ICU-positive delirium. Nonlinear algorithms did not improve discrimination. External results were promising but imprecise and require confirmation in larger cohorts with protocolized delirium assessment before clinical use.