Preprint
Aug 2026
MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning
Experiments on GPT-2 Small, Medium, and Large pretraining show that the proposed methods outperform Muon while maintaining nearly the same memory footprint and wall-clock time.
Tongle Wu, Huanyu Dong, Ying Sun et al.
· 1 citation