#artificial intelligence
May 2026
Demystifying Manifold Constraints in LLM Pre-training
By controlling the update-to-weight ratio, constrained training significantly alleviates update cancellation, empirically demonstrating that MACRO is robust to low-precision computation and competitive with existing algorithms for standard LLM pre-training.
Kang An, Jiaxiang Li, Donald Goldfarb et al.
· arXiv.org · 1 citation