Deep Learning-Based Thyroid Disease Prediction with Fairness-Aware Evaluation: A Comparative Study Against Classical Machine Learning Approaches

Abstract

Thyroid disease prediction using machine learning has been extensively studied, with existing literature dominated by classical algorithms such as Random Forest and XGBoost, often achieving accuracies exceeding 98% through careful imbalance handling and feature selection. However, two gaps remain: deep learning architectures are rarely benchmarked as primary models on this task, and no existing study examines whether model performance holds consistently across patient subgroups. This paper addresses both gaps using the UCI Thyroid Disease dataset (n = 8,865). We benchmark three classical models (Logistic Regression, Random Forest, XGBoost) against two deep learning architectures (a multilayer perceptron and TabNet, an attention-based tabular deep learning model), using threshold-matched comparison to ensure fair evaluation. XGBoost achieved the strongest overall performance (ROC-AUC = 0.990), outperforming both deep learning models even after recall calibration. We then conducted a subgroup fairness analysis across patient age and sex. While no significant disparity was found by sex, a statistically significant age-based disparity emerged: recall for patients under 30 was substantially lower than for patients 70 and older (XGBoost: 85.9% vs. 97.0%, χ² = 5.87, p = 0.015; TabNet showed a larger 14.5 percentage point gap). A cost-sensitive analysis weighting missed diagnoses more heavily than false alarms showed per-patient clinical cost was over three times higher for younger patients. These findings suggest that while classical ensemble methods remain difficult to outperform on tabular thyroid data, both classical and deep learning models exhibit a clinically meaningful age-based bias that warrants attention before deployment in screening contexts.