Comparing LightGBM and Deep Neural Networks for Early Diabetes Prediction
TL;DR
LightGBM is a more dependable option in situations where data are limited and interpretability is important, at least for this type of structured tabular input, according to a publicly available Kaggle dataset.
Abstract
Currently, diabetes affects around 589 million adults worldwide and is one of the more pressing chronic disease burdens in modern public health. Machine learning is widely accepted in diabetes risk modeling, although most published work tends to evaluate models in isolation rather than using them side by side in matching conditions. In particular, how LightGBM is compared to simple deep neural network structures in small, structured clinical datasets is not well described. This study directly addressed this question using a publicly available Kaggle dataset of 1,879 patient records and 46 clinical variables. The paper evaluated two models from three dimensions: differentiation, calibration, and interpretability. LightGBM is better overall than the DNN --AUC 0.965 vs. 0.913, accuracy 0.941. 0.870 – resulting in a significant reduction in missed diabetes cases. To explore what the model actually relies on, it applied SHAP analysis; fasting blood glucose and HbA1c were key drivers, consistent with clinicians' expectations. Overall, the results suggest that LightGBM is a more dependable option in situations where data are limited and interpretability is important, at least for this type of structured tabular input.