Cross-Cohort Reliability of Machine Learning for Breast Cancer Survival Prediction

Abstract

Transportability limitations can affect discrimination, probability calibration, and uncertainty estimates even when machine-learning pipelines use the same input variables. This study evaluated 5-year breast cancer survival models across METABRIC and TCGA-BRCA using age and ER, PR, and HER2 status. Logistic regression, radial-basis-function support vector machine, random forest, XGBoost, and soft voting underwent bidirectional external validation. The METABRIC soft-voting model was further assessed with source-only Platt scaling, split conformal prediction, selective prediction by singleton sets, cohort classification, and exploratory few-shot target adaptation. The principal contribution is an integrated reliability framework. METABRIC-to-TCGA ROC-AUC decreased numerically from 0.732 to 0.643, an observed difference of approximately 0.088 with a 95 percent confidence interval from minus 0.015 to 0.196. TCGA-to-METABRIC ROC-AUC decreased numerically from 0.674 to 0.586, also by approximately 0.088, with a 95 percent confidence interval from minus 0.127 to 0.282. Neither interval excluded zero. Source Platt scaling improved the observed internal Brier score and expected calibration error but produced worse observed values for both metrics externally. TCGA empirical conformal coverage was 81.8 percent versus nominal 90 percent. Exploratory target recalibration improved mean locked-test Brier score and expected calibration error at larger target sample sizes, whereas updating only the conformal threshold did not restore nominal mean coverage. These findings support separate evaluation of ranking, probability accuracy, and uncertainty under cohort shift and do not establish model superiority, clinical usefulness, or deployment readiness.