Learning sparse mixture‐of‐experts generalized linear models in ultrahigh dimensions

Abstract

Motivated by the challenges of heterogeneity and high dimensionality in the analysis of modern data, we investigate continuous regularization methods for learning sparse mixture‐of‐experts generalized linear models (MoE‐GLM). Although there are foundational results about regularized estimators in a broad class of regression models including GLMs, to the best of our knowledge, there is no general theoretical result on consistency in estimation and feature selection of these estimators in MoE‐GLM under ultrahigh‐dimensional settings. We address this gap by studying a general class of regularized estimators for sparse MoE‐GLM. Our results also apply to finite mixtures of regressions, a subclass of MoE‐GLM. The methods are implemented using a modified expectation‐maximization algorithm combined with proximal coordinate descent. We evaluate the empirical performance of the methods via simulations, and demonstrate their practical use with a real data analysis.