#machine learning
Oct 2025
Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noise
The theoretical results show that GN's optimality for linear regression no longer holds under a poorly chosen basis under both population and stochastic updates, or a move from linear regression to a non-convex variant of logistic regression, and Adam-style methods offer genuine advantages over curvature-inspired preco...
Bingbin Liu, Rachit Bansal, Depen Morwani et al.
· arXiv.org · 3 citations