Deadline-Aware Adaptive Prefill Chunking for Efficient Large Language Model Serving
This work introduces SLOWeave, an online scheduling method that selects the largest prefill chunk predicted to finish before the earliest active decode deadline, and proves that, whenever a decode-only iteration is feasible and the cost predictor is accurate, SLOWeave maximizes immediate prefill progress among decision...
Si-Yu Song, Qi-Wei Bai, Jin-Bo Hao et al.
· 1 citation