The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization
It is shown that vanilla SGDA, without any modification to its update rule, can converge under heavy-tailed noise in both nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C) settings, establishing the first convergence guarantees for SGDA in these regimes.
Tian-Xiu Zhu, Yi Xu, Xiang-Yang Ji
· 0 citations