From Fairness Findings to Fairness Claims: An Evidence Classification Scheme for Clinical AI

Abstract

Fairness audits of clinical AI models rarely make the evidentiary status of subgroup findings explicit: reassuring results may reflect insufficient statistical precision rather than true parity, and audit verdicts can easily reverse under equally defensible analytic choices. We introduce an evidence classification scheme that screens for sample size and precision, and integrates stability across design alternatives directly into the fairness claim. We demonstrate this scheme on the estimation of the brain-age gap (BAG), a potential clinical biomarker, from structural MRI using the Alzheimer's Disease Neuroimaging Initiative (ADNI) data. The male-female and Black-vs-White differences, along with the White-Male and Black-Female intersectional contrasts, are all classified as equivalence supported, stable across regressor choice (ridge vs. gradient-boosted trees) and feature representation (full feature set vs. cortical-thickness-only). The Asian-vs-White and Black-Male comparisons remain classified as insufficient data throughout, as neither meets the pre-specified minimum-sample threshold. The proposed scheme provides a path from raw fairness findings to justified fairness claims via pre-specified thresholds, minimum-information screening, and stability checks across declared design choices.