Enhancement of Elastic-net Model via Stochastic Gradient Descent and Adam Optimization: Application to Prostate Cancer Data
TL;DR
Simulated data analysis showed that both Adam and SGD Elastic-net models produced more precise coefficient estimates with narrower confidence intervals, improving interpretability and robustness, and achieved faster convergence and lower residual errors, making them highly effective for high-dimensional datasets facing multicollinearity.
Abstract
High-dimensional data and multicollinearity present major challenges in regression analysis, often causing overfitting and unstable coefficient estimates. The Elastic-net model, which combines (Lasso) and (Ridge) regularization, offers a robust solution by enhancing feature selection and handling multicollinearity. This study improves Elastic-net by integrating two optimization techniques: Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (Adam). The SGD Elastic-net model accelerates convergence and boosts computational efficiency through mini-batch updates, while the Adam Elastic-net model incorporates adaptive learning rates and momentum to enhance stability and performance, especially with noisy or sparse data. Simulated data analysis showed that both Adam and SGD Elastic-net models produced more precise coefficient estimates with narrower confidence intervals, improving interpretability and robustness. A real-world application on a prostate cancer dataset further confirmed these advantages. Diagnostic plots validated the assumptions of linearity, normality, and homoscedasticity, supporting model reliability. Overall, the Adam and SGD Elastic-net approaches achieved faster convergence and lower residual errors, making them highly effective for high-dimensional datasets facing multicollinearity.