A Ranked Sparsity Extension to the Bayesian Information Criterion: A Tool for Selecting Variables from Multiple Data Modalities
The concept of ranked sparsity, originally introduced in the context of penalized regression, arises in modeling applications when an expected disparity exists in the quality of information between different feature sets. Its presence can cause traditional and modern model selection methods to fail because such procedures commonly presume “covariate equipoise”—that each potential parameter is equally worthy of entering into the final model. However, this presumption does not always hold, especially in the presence of derived variables or with highly disparate feature sets (i.e., multi-modal data). For instance, when all possible interactions are considered as candidate predictors, the sheer number of them grossly inflates the number of false discoveries, resulting in unnecessarily complex and difficult-to-interpret models with many (truly spurious) interactions. In this work, we motivate a ranked sparsity extension to the Bayesian Information Criterion (RBIC) that requires a stronger level of evidence in order to allow certain variables (e.g., interactions vs main effects and genetic vs clinical covariates) into a model. We compare the performance of RBIC relative to competing methods for selecting polynomials and interactions in a simulation study and in two applications, showing that stepwise selection guided by RBIC produces better-predicting, more transparent models (with fewer false interactions) compared to existing alternatives.