#artificial intelligence
Sep 2026
Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition
Experiments show that GDLoRA consistently improves over LoRA and narrows the performance gap to FFT, and incorporates complementary normal gradients without increasing standard LoRA's optimizer-state memory budget under matched adapter and optimizer configurations.
Yi Ouyang, Shi-Wei Li, Hao-Zhao Wang et al.
· arXiv.org · 0 citations