Wen-Qing Su

Efficient sparse low-rank adaptation for large language models.

Parameter-efficient fine-tuning (PEFT) methods adapt foundation models to specific domains by selectively updating only a small subset of critical parameters, significantly reducing computational costs. Among these methods, Sparse Low-Rank Adaptation (SoRA) has emerged as an effective approach for fine-tuning large lan...