Preprint
Aug 2026
Beyond Binary Priorities: Multi-Tier SLA Scheduling for Large Language Model Serving
This work extends Llumnix's priority model to support an arbitrary number of tiers and evaluates the effects of this extension under three realistic priority distributions (uniform, Gaussian, enterprise) using Vidur, a high-fidelity LLM inference simulator.
Anders Vestrum, Arya Raeesi, Hanna Roed
· 0 citations