Balancing and Beyond: Communication-Centric Optimizations in Expert Parallelism
EPIC mitigates imbalance via performance-aware expert migration and runtime expert activation, and then improves communication with topology-adaptive transport kernels and fine-grained computation-communication overlap.
Jiamin Cao, Qingxu Li, Yaozhong Liu et al.
· Proceedings of the ACM SIGCO... · 0 citations