Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise
It is proved that SGD with clipping and additive Gaussian noise (SGD-CN) converges almost surely (a.s.) under smoothness and uniformly bounded stochastic-gradient noise assumptions, provided the step sizes satisfy some standard decaying conditions.
Amartya Mukherjee, Jun Liu
· 0 citations