StageML: Partial Evaluation for Multi-Tenant MoE-LoRA Inference
StageML, a partial evaluation (PE) system that exploits the temporal boundaries between deployment, adapter loading, tenant request admission, expert routing, and token generation using a binding-time lattice, and the results show that PE complements low-level kernel optimization.
X. Adettu, Tian Zhao
· 0 citations