NeuroFuzzyAdam: A Fuzzy-Enhanced Adaptive Optimization Algorithm for Deep Learning

Abstract

Deep neural networks are commonly trained using adaptive optimization methods such as Adam because they converge quickly and perform well under stochastic training conditions. Despite these advantages, recent research has revealed several important drawbacks of Adam. In particular, the optimizer tends to converge toward sharp minima, exhibits oscillatory behavior during loss optimization, and can produce unstable parameter updates when training in highly noisy environments. In this work, we propose NeuroFuzzyAdam (NF Adam), a novel optimizer that integrates fuzzy logic-based scalar modulation directly into the Adam update rule to improve learning stability and generalization. By introducing a bounded correction term via hyperbolic tangent transformations, NF Adam adaptively regulates step sizes based on moment estimates while preserving Adam’s key strengths. We provide theoretical convergence guarantees under convexity and bounded-gradient assumptions, matching Adam’s O(1/T) regret bound. Empirically, we evaluate the NF Adam across both image (MNIST, Fashion-MNIST, CIFAR-10) and tabular (UCI Iris, Wine, Breast Cancer) datasets using consistent architectures. Results show that NF Adam improves training smoothness, reduces gradient norm variance, and achieves competitive classification accuracy compared to Adam, SGD, RMSprop, and AdamW. These findings suggest that incorporating fuzzy logic into optimization dynamics presents a promising direction for enhancing robustness in neural network training, especially in scenarios with high gradient noise or non-smooth loss surfaces.