Ziqing Wang

Soft Orthogonal Low-Rank Adaptation for Knowledge Sharing in Large Language Model Continual Learning

This work proposes At-tribution Scores-based Soft Orthogonality Low-Rank Adaptation (ASO-LoRA), an effective and efficient framework that simultaneously facilitates knowledge transfer while mitigating catastrophic forgetting and achieving a balance between knowledge transfer and preservation.