Efficient and Interpretable Mixtures of Experts: Statistical Inference, Initialization, and Applications
This work develops a supervised-compression initializer that provides a data-driven initial guess for the number of clusters and proposes likelihood-based procedures for confidence intervals and a fixed-gate bootstrap method for prediction intervals, demonstrating that MoE models can serve as a flexible, interpretable,...