Sep 2026
Learning Without Forgetting for Continual Learning in LLMs Through Adaptive LoRA Ranks
A novel parameter-efficient approach for continual learning in LLMs is proposed, which adaptively adjusts layerwise LoRA ranks to help balance the stability–plasticity trade-off, and leverages a subspace similarity metric to evaluate the orthogonality between task-specific low-rank subspaces.
Fuli Qiao, Mehrdad Mahdavi
· Machine-mediated learning · 0 citations