Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature
Low-Rank Adaptation (LoRA) is an effective approach for adapting large pretrained models by learning low-rank weight updates. In practice, the LoRA rank is used to control an adapter's parameter budget and representational capacity. We show that this view is incomplete: while the nominal rank determines the representat...
Zi-Han Zhu, Zhe-Hang Du, Xu-Yang Chen et al.
· 0 citations