Task-Aware Federated Fine-Tuning for MoE-based Large Language Models
Mixture-of-Experts (MoE) has become a widely adopted architecture for Large Language Models (LLMs), as it improves model capacity while limiting computational overhead through sparse expert activation. This property makes MoE-based LLMs particularly attractive for resource-constrained distributed environments. However,...
Ting-Qi Wang, Hongyu Ke, Hao-Xin Wang et al.
· 0 citations