Sanjali Yadav

Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging

Mixture-of-Experts (MoE) models are a compelling architecture for scaling model capacity, making them especially attractive for deployment on resource-constrained, single-GPU systems. However, this benefit is difficult to realize because expert parameters dominate memory, and token-level routing is dynamic, unpredictab...