UNAS: Urgency- and Fairness-Aware Scheduling for SLO-Oriented LLM Serving

Abstract

The widespread adoption and strong generalizability of large language models (LLMs) lead to highly heterogeneous workloads that exhibit substantial variability in request lengths and latency requirements. This pronounced heterogeneity causes existing scheduling strategies to suffer from head-of-line blocking and inefficient request ordering, resulting in significant SLO violations and inflated end-to-end latency. In this paper, we present UNAS, a hierarchical scheduler for end-to-end SLO-oriented LLM serving. UNAS adopts a two-level design. At the cluster level, it classifies incoming requests into urgent and normal queues based on estimated service times derived from request features and runtime statistics, and dynamically updates their queue priorities as latency pressure evolves. At the instance level, UNAS performs fine-grained dispatch using a least-slack-first policy, prioritizing requests with the highest risk of SLO violations. Experimental results demonstrate that UNAS consistently outperforms representative baselines across diverse workloads, improving end-to-end SLO attainment by up to 11.1% and reducing mean end-to-end latency by up to $1.47 \times$.