Enhancing local robustness with Hölder regularization.

Abstract

Deep neural networks (DNNs) are vulnerable to small input perturbations, resulting in instability and adversarial susceptibility. While adversarial training methods such as PGD-AT and TRADES improve robustness by generating worst-case adversarial examples, they do not explicitly regulate the model's local smoothness. In this paper, we propose Hölder Regularization (HR), a novel method to enhance local robustness by constraining the model's output variations based on Hölder continuity. Unlike conventional adversarial defenses, HR directly controls the functional smoothness of the model by regulating the relationship between input distances and output variations. We provide theoretical insights showing that the proposed regularization directly tightens an upper bound on the adversarial risk, thereby improving robustness. We also demonstrate that HR can be effectively combined with adversarial training, offering a complementary approach to existing techniques. We further compare HR with representative robustness baselines, including adversarial training methods and spectral normalization, showing that HR achieves superior or competitive performance across different perturbation settings. Experiments on MNIST, CIFAR-10, BSEQ, and ISOLET show that our method consistently improves robustness under PGD attacks, Gaussian noise, ℓ∞-perturbations, and feature masking. Furthermore, the proposed approach remains computationally efficient, as the Hölder regularization term is approximated via mini-batch sampling, introducing only a modest overhead.