FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization
Federated Learning (FL) enables privacy-preserving fine-tuning of Large Language Models (LLMs), yet the massive communication overhead remains a critical bottleneck. Furthermore, applying Low-Rank Adaptation (LoRA) in FL faces a fundamental"aggregation dilemma"between the accurate Sum-of-Products (SoP) and the communic...
Han Zou, Chao Zhang, Yu-Zhi Yang et al.
· 0 citations