Understanding the Subspace Stabilization of the Hessian and Gradient Covariance Matrix
In the gradient flow set-up, it is shown that the slow evolution of the idenfied approximation is due to the separation between the outlier and the bulk eigenvalues of the Hessian matrix, thus providing an explanation to the phenomenon of the top subspace stabilization of the Hessian matrix.
Fang-Shuo Liao, Anastasios Kyrillidis
· 0 citations