Yashas Malur Saidutta

Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks

Net Utility is introduced, a data free metric that first decomposes every task LoRA by its Singular Value Decomposition (SVD) and scores each of those singular directions by its task utility and its interference with other tasks directions, which shows that net utility based rank allocation outperforms its counterparts...