Sustainable AI Request Scheduling with Joint Compute, Network, and Power Optimization
TL;DR
RAPID is proposed, a region-aware and power-informed scheduling framework that integrates static and online heuristic schedulers for large-scale AI request scheduling that significantly reduces carbon emissions, electricity costs, and total energy consumption.
Abstract
The rapid growth in artificial intelligence (AI) demand has significantly increased the electricity consumption and carbon emissions of computing centers. How to schedule AI requests across computing centers to reduce carbon emissions and electricity costs while maintaining low latency is an essential research problem. Existing schedulers reduce emissions by shifting workloads or balancing resources but usually simplify power system modeling, ignore transmission-side costs and carbon emissions, or make local decisions without batch-level coordination. To better address these problems, we first develop an ILP-based scheduler to get optimized results, but it faces scalability limitations. Then, we propose RAPID, a region-aware and power-informed scheduling framework that integrates static and online heuristic schedulers for large-scale AI request scheduling. Experiments based on real-world GenAI traces and Chinese regional power profiles show that RAPID significantly reduces carbon emissions, electricity costs, and total energy consumption compared to methods from previous works while maintaining zero Service Level Agreement (SLA) violations.