Preprint
Oct 2026
Hybrid Momentum Methods for Optimizing Deep Learning Models
This work proposes Hybrid Momentum (HyM), which combines a linearly weighted average of past gradients with a geometrically discounted fast momentum, and proposes HyMUON, which applies Newton--Schulz orthogonalization to the hybrid direction.
Hai-Feng Xu, Yi-Jin Ren, Chenyu Zhou et al.
· 0 citations