Zi-Jian-Ai-Yan-Di-Chen-Hong-Yu Liu

Parameter-Efficient Fine-Tuning of BERT: A Study of Fixed-Rank and Layer-Wise LoRA on SST-2

Parameter-efficient fine-tuning (PEFT) has emerged as a crucial method for adapting pretrained Transformer models to downstream tasks without the need to fine-tune the full set of model parameters. Low-Rank Adaptation (LoRA) is a technique that reduces the number of trainable parameters by representing weight updates w...