Shape: Shaping The Low-Rank Subspace With Hierarchical Priors
The escalating scale of Large Language Models (LLMs) has made Parameter-Efficient Fine-Tuning (PEFT) an essential paradigm for localized adaptation. However, standard methods like Low-Rank Adaptation (LoRA) typically rely on a static and uniform rank distribution across all layers. This structural agnosticism neglects...