Adversarial LassoNet is proposed, a stability-driven sparse feature selection framework that integrates input-space adversarial perturbations with the hierarchical sparsity mechanism of LassoNet and an NTK-inspired spectral analysis to characterize how perturbation-driven training can reduce gradient concentration.
Abstract
Sparse feature selection is critical for high-dimensional machine learning, yet traditional $\ell_1$-regularized methods are often brittle under observational noise and spurious correlations, leading to unstable feature supports and degraded generalization. Although adversarial training has been widely used to improve model robustness, its interaction with hierarchical sparse feature selection remains underexplored. In this work, we propose Adversarial LassoNet (AdLNet), a stability-driven sparse feature selection framework that integrates input-space adversarial perturbations with the hierarchical sparsity mechanism of LassoNet. We derive a tractable first-order adversarial approximation under local smoothness assumptions and provide an NTK-inspired spectral analysis to characterize how perturbation-driven training can reduce gradient concentration. Experiments on high-dimensional SERS data, six public benchmark datasets, and ColoredMNIST show that AdLNet maintains competitive sparse-selection performance while improving out-of-distribution robustness by 4.4\% and feature support reproducibility by 6.3\% under nearly matched support sparsity on ColoredMNIST. On the high-dimensional lung cancer screening dataset, AdLNet achieves a 5.3\% test accuracy gain and a 6.0\% AUC improvement over vanilla LassoNet. Code and dataset are available at https://github.com/719573/Adversarial-LassoNet.
Mutual Heterogeneous Learning (MHL) is proposed, a framework enabling robust pruning via single-model inference that significantly outperforms single-model baselines in both adversarial robustness and corruption robustness, while maintaining competitive clean accuracy.
Jinhui Yu, Zikai Zhang, Khaled A. Harras et al.· 0 citations
A consistency theory for robust neural classifiers based on the S-divergence family that requires no identifiability assumption is developed and it is proved that empirical S-divergence minimizers converge to the population-optimal equivalence class under mild regularity conditions.
Subhabrata Majumdar, Anand Deo, Partha Pratim Saha et al.· 0 citations
In real-world scenarios, the training data usually contains redundant features, label noise and feature noise, which provide severe challenges for the efficiency of machine learning methods. Since standard support vector machine (SVM) adopts $l_2$-norm penalty and hinge loss function, it lacks the ability of selecting significant features and is sensitive to noise. To address these issues, this paper proposes a novel asymmetric, robust, bounded, sparse and smooth (aR) loss function for $l_1$-norm penalized geometric twin SVM (aRSGTSVM) to handle classification and regression tasks. The $l_1$-norm penalty can achieve the feature selection. The proposed aR loss function can not only effectively mitigate the impact of label noise, but also significantly enhance the stability to resampling noise, i.e., the zero-mean feature noise around the boundary hyperplanes. Furthermore, a statistical analysis of the robustness of aRSGTSVM was also conducted using the influence function. Since aRSGTSVM involves nonconvex and nonsmooth optimization, we develop a fast and stable proximal gradient descent based solving algorithm. Compared with related state-of-the-art methods, experimental results demonstrate the superiority of the proposed aRSGTSVM on both synthetic and UCI datasets. Furthermore, we apply aRSGTSVM to index tracking tasks, where results for tracking the different indices in the China stock market show that it can achieve satisfactory performance.
PILO is established, a new, more effective paradigm for robust transfer learning through principled and targeted parameter optimization, and significantly outperforms state-of-the-art full-parameter and parameter-efficient methods in robust accuracy across multiple benchmarks.
Shuaihe Liu, Qiugang Zhan, Guisong Liu et al.· 0 citations
Adversarial training has emerged as a leading strategy for enhancing the robustness of machine learning models against adversarial attacks. Its effectiveness often wanes when faced with unseen adversarial examples, resulting in suboptimal robust generalization. To address this issue, we introduce a novel energy-based optimization strategy to improve the robust generalization by incorporating the principles of energy-based models. Our framework models the energy of natural and adversarial examples, where natural samples are assigned to lower energy and adversarial samples to higher energy. During the inference phase, the influence of adversarial perturbation can be alleviated by energy minimization. Theoretically, we show that the proposed energy-based optimization strategy yields a tighter robust-generalization bound through an explicit energy-discrepancy term; this analysis provides an explanatory bound and should not be interpreted as certified robustness. Empirically, a series of evaluations provide evidence for the efficacy of the proposed methodology under the specified threat models and evaluation protocols, showing strong and competitive performance across three extensively utilized datasets. Specifically, EM-AT achieves 77.71% standard-AA robustness on CIFAR-10 and remains highly competitive under comparable lightweight settings. The source codes are available at https://github.com/LitterQ/EM-AT.
The increasing scale of deep neural networks has necessitated model compression techniques, with pruning emerging as a prominent approach to reduce computational and memory costs. However, aggressive pruning introduces a critical challenge: the degradation of mathematical stability and adversarial robustness. Recent research reveals that highly pruned weight matrices tend to become ill-conditioned, exhibiting exploding condition numbers that undermine model performance and robustness . This paper proposes a condition number-aware pruning framework that explicitly preserves mathematical stability during the pruning process. We establish theoretical connections between sparsity, condition number, and local Lipschitz continuity, demonstrating that the condition number becomes the dominant factor limiting robustness in over-sparsified models . Our methodology integrates a differentiable Condition Number Constraint (CNC) with transformed sparse regularization (TSCNC) to simultaneously achieve high sparsity and well-conditioned weight matrices. Experimental evaluations on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that our approach significantly improves both standard accuracy and adversarial robustness compared to conventional pruning methods, achieving superior performance across VGG, ResNet, and WideResNet architectures.
Jeromy R, K Bhavani, K.Mohana Lakshmi et al.· International journal of com...· 0 citations