External validation of machine learning models for predicting prehospital delay in acute ischemic stroke: a retrospective cohort study
Abstract
Background and purpose Prehospital delay remains a major barrier to timely reperfusion therapy in acute ischemic stroke (AIS). We externally validated eight prediction algorithms and examined whether complex machine-learning models added value beyond logistic regression. Methods This retrospective cohort study included consecutive patients with AIS from two hospitals in Kashgar, China (January 2019–December 2024). Hospital 1 (n = 1,377) was the development cohort and Hospital 2 (n = 1,015) the independent validation cohort. Prehospital delay was defined as onset-to-door time >4.5 h. LASSO selection and tuning were restricted to Hospital 1. Sensitivity analyses used all 36 encoded candidate features and repeated all models without SMOTE. Evaluation included discrimination, calibration, post-hoc recalibration, and decision curves. Results Delay occurred in 80.9 and 83.2% of the two cohorts. In the prespecified LASSO-plus-SMOTE analysis, XGBoost had the highest validation AUC (0.771; 95% CI, 0.734–0.807), but did not significantly outperform logistic regression (difference, 0.015; 95% CI, −0.003 to 0.034; p = 0.107). Full-feature modeling produced no consistent gain. Omitting SMOTE improved calibration and often discrimination; logistic regression without SMOTE achieved AUC 0.774 and Brier score 0.122, versus 0.756 and 0.180 with SMOTE. The original SMOTE-trained logistic model underestimated absolute risk. Rural residence, non-emergency-channel presentation, referral, and non-ambulance transport were associated with absolute risk differences of 15.8, 21.5, 9.2, and 36.3 percentage points in Hospital 2. Conclusion Complex algorithms did not show a clear advantage over logistic regression. Because several predictors are available only after arrival, the framework is intended for retrospective health-system profiling and quality-improvement planning, not individual pre-arrival prediction. Universal interventions remain warranted given the high prevalence of delay.