Yi-Da Wang

vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference

Mixture-of-Experts (MoE) models have become prevailing, yet their distributed deployment suffers from severe dynamic load imbalance due to the conflict between static expert placement and inherent routing dynamism. While host DRAM offers capacity to avoid static device storage for load balancing, we identify existing o...