Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
This work establishes a theoretically sound and empirically proven approach for resolving memory-constrained scheduling in modern LLM deployments by introducing the Smallest Volume First (SVF) algorithm and its highly efficient variant, 1-bit SVF.