Anders Vestrum

Beyond Binary Priorities: Multi-Tier SLA Scheduling for Large Language Model Serving

This work extends Llumnix's priority model to support an arbitrary number of tiers and evaluates the effects of this extension under three realistic priority distributions (uniform, Gaussian, enterprise) using Vidur, a high-fidelity LLM inference simulator.