Qi-Wei Bai

Deadline-Aware Adaptive Prefill Chunking for Efficient Large Language Model Serving

This work introduces SLOWeave, an online scheduling method that selects the largest prefill chunk predicted to finish before the earliest active decode deadline, and proves that, whenever a decode-only iteration is feasible and the cost predictor is accurate, SLOWeave maximizes immediate prefill progress among decision...