A Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay
A broad class of neural networks trained under the $\ell^2$ loss by gradient descent with weight decay with weight decay is studied, and the convergence of GD to a neighbourhood of the global minimizers of the empirical loss is proved.
Yu-Qing Wang, I. Kevrekidis, Mikhail Belkin
· 1 citation