Beyond Binary Classification: A Comprehensive Survival Analysis and Calibration Assessment of the Heart Failure Clinical Records Datase

Abstract

Background: The Heart Failure Clinical Records dataset (n = 299) is among the most extensively reused benchmark datasets in clinical machine learning, yet the overwhelming majority of published analyses treat patient mortality as a static binary classification target and rarely test the statistical assumptions underlying their models. Methods: We instead modeled mortality as a time-to-event outcome using Kaplan-Meier estimation, multivariable Cox proportional hazards (PH) regression, and Random Survival Forest (RSF). We explicitly tested the proportional hazards assumption, evaluated discrimination using time-dependent AUC, assessed calibration via the integrated Brier score, and examined model performance across sex, age, and diabetes subgroups. Results: Age, ejection fraction, and serum creatinine emerged as the dominant predictors of mortality across both Cox PH and RSF models. Ejection fraction violated the proportional hazards assumption (p = 0.013); after stratifying by ejection fraction category, RSF outperformed the corrected Cox model on an identical hold-out set (C-index 0.775 vs. 0.716). RSF achieved a mean time-dependent AUC of 0.820, but calibration deteriorated sharply in late follow-up (Brier score rising to 0.53 near day 270), despite high discrimination in that same window. No sex-based survival difference was observed (log-rank p = 0.950), and subgroup C-index values were broadly comparable across sex, age, and diabetes status, though small subgroup sizes limit precision. Conclusions: Framing this dataset as a survival problem reveals assumption violations, calibration failures, and confounding effects that are invisible to standard binary classification pipelines, even though the same core clinical predictors are recovered. We recommend that future studies on this dataset report proportional hazards diagnostics and calibration alongside discrimination metrics.