Inverse-based Lasso Regression: Novel Algorithms for Feature Selection and Multicollinearity Mitigation

Abstract

As data accumulation continues to expand and information technologies evolve, machine learning methods have become widely adopted, making the effectiveness of learning algorithms crucial. Among the most popular machine learning models is Lasso regression, renowned for its feature selection capabilities and ability to address multicollinearity. This paper introduces novel algorithms for estimating Lasso regression parameters by reformulating the problem as an inverse single-point optimization task. Two algorithms are proposed: Lasso-I, which implements coordinate descent with L1 regularization, and Lasso-H, a hybrid approach that combines Lasso-I with wrapper techniques for feature selection using information criteria. The iterative algorithms involve calculating partial derivatives and selecting arguments for adjustment based on residual sum of squares or information criteria. Algorithm evaluation was performed using linear and logistic regression models across diverse datasets from KEEL and UCI repositories, alongside various metrics including the AIC, MSE, and R 2 . The experimental results demonstrate that the algorithms effectively address parameter estimation problems, with Lasso-H achieving optimal AIC values in 90% of logistic regression cases. The proposed methods eliminate the need for explicit regularization parameter specification while maintaining robust feature selection capabilities and effective multicollinearity mitigation, demonstrating high accuracy and reliability across high-dimensional datasets.