#machine learning
Sep 2025
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
Low-rank orthogonalization is proposed, which performs orthogonalization by leveraging the low-rank nature of gradients during NN training by leveraging the low-rank nature of gradients during NN training.
Chuan He, Zhanwang Deng, Zhaosong Lu
· arXiv.org · 20 citations
· ⚡2