Richard Mortier

ALPS: An Application-Aware, Length-Predictive Scheduler for Efficient LLM Inference at Edge

The deployment of Large Language Models (LLMs) on resource-constrained edge server clusters introduces a critical systems challenge: achieving high throughput and low latency under strict GPU memory and bandwidth limits. Current schedulers are ill-suited for this environment, as they treat requests independently, incur...