Preprint
Aug 2026
Global Simulation-Guided Dynamic Operator Scheduling for Efficient Multi-Tenant Model Serving
Container-granularity scheduling leaves abundant short-lived idle slices within containers unexploited. Reallocating containers is too heavyweight to utilize such fine-grained opportunities under SLA constraints, and operator-level scheduling requires reasoning about dependencies, memory safety, and cluster-wide execut...
Weinan Liu, Zeyuan Ding, D. Ding et al.
· 0 citations