Development of interpretable machine learning models for predicting the probability of sepsis in patients with pulmonary fibrosis in the intensive care unit: based on MIMIC-IV and multi-database validation

Abstract

Background Limited by small sample size, single-institution design, and insufficient comprehensive external validation across heterogeneous healthcare systems, no study to date has systematically validated the predictive performance of machine learning models for sepsis occurrence in an intensive care unit (ICU) population with concomitant pulmonary fibrosis through multiple large-scale databases. Methods This retrospective multi-database study utilized two large databases to establish and validate a machine learning model for predicting the probability of sepsis occurrence in ICU patients with pulmonary fibrosis. In this study, 542 patients from the MIMIC-IV database were divided into a training set (381 patients) and an internal validation set (161 patients) in a 7:3 ratio, and external validation was performed on the MIMIC-III (186 patients) database. Six machine learning algorithms were employed: Decision Tree (DT), Extreme Gradient Boosting (XGBoost), Logistic Regression (LR), Lightweight Gradient Boosting Machine (LightGBM), Support Vector Machine (SVM), and Artificial Neural Network (ANN). Baseline variables were screened using least absolute shrinkage and selection operator (Lasso) regression to identify potential predictors. The interpretability of the model was evaluated using Shapley Additive Explanations (SHAP) analysis. Results The entire cohort consisted of 728 ICU patients with pulmonary fibrosis. We identified nine consistently crucial clinical characteristics, including gender, dementia, pneumonia, antibiotics, nephrotoxic drugs, glucocorticoids, sequential organ failure assessment (Sofa) score, red blood cell distribution width, and total serum calcium. The ANN algorithm performed optimally, with an area under the curve (AUC) of 0.878 in the training set, 0.837 in the internal validation set, and 0.857 in the MIMIC-III external validation set. SHAP analysis indicated that Sofa was the most influential predictor, followed by antibiotics and pneumonia. Additionally, a web tool was developed to facilitate the prediction of sepsis probability in clinical practice. Conclusions This study is the first to develop and validate a machine learning model for predicting sepsis in ICU patients with pulmonary fibrosis across multiple databases. The ANN model, combined with SHAP interpretability, provides a reliable decision-making tool for clinical decision support, and its consistency has been verified in two databases, including our internal validation cohort.