Optimization of glucocorticoid administration in patients with sepsis using reinforcement learning: a multicenter retrospective study
The use of glucocorticoids in sepsis remains controversial due to heterogeneous treatment responses across patient populations. More individualized treatment strategies are needed to guide glucocorticoid administration. An offline reinforcement learning (RL) model was developed using the MIMIC-IV database and externally validated in the eICU-CRD. Adult patients with sepsis were included, and clinical trajectories were constructed using 24-hour time steps. Least absolute shrinkage and selection operator (LASSO) regression was applied for feature selection, and missing data were handled using multiple imputation. The problem was formulated as a Markov decision process, with glucocorticoid dosing defined as discrete actions. A conservative Q-learning (CQL) algorithm was used to learn treatment policies. Policy performance was evaluated using fitted Q-evaluation (FQE), and comparisons were made with clinician strategies. SHAP analysis was performed to interpret model decisions. A total of 3,070 patients from the MIMIC-IV cohort and 372 from the eICUCRD cohort were included. The CQL policy achieved higher expected returns than clinician policy in both datasets. Higher expected returns were associated with higher survival rates. The model recommended a more selective glucocorticoid use pattern, with reduced overall usage and delayed initiation in some patients. The RL model identified individualized glucocorticoid treatment strategies and demonstrated improved policy performance compared with clinician practice, suggesting its potential as a clinical decision support tool.