Ru-Bing Yang

The Dynamics of Generalization in Deep Learning

This work derives a differential equation that governs the evolution of the generalization gap when a model is trained by gradient descent-based methods and shows that the generalization gap is given by a quadratic form that consists of an ``effective Gram matrix'' that depends upon the training trajectory and a certai...