This work introduces a novel LLM-based predictive scheduling system designed to enhance operational efficiency while reducing the environmental impact of data centers, using an LLM to predict key metrics such as execution time and energy consumption from source code.
Abstract
The growing demand for AI-driven workloads, particularly from Large Language Models (LLMs), has raised concerns about the significant energy and resource consumption in data centers. This work introduces a novel LLM-based predictive scheduling system designed to enhance operational efficiency while reducing the environmental impact of data centers. Our system utilizes an LLM to predict key metrics such as execution time and energy consumption from source code, and it has the potential to extend to other sustainability-focused metrics like water usage for cooling and carbon emissions, provided the data center can track such data. The predictive model is followed by a real-time scheduling algorithm that allocates GPU resources, aiming to improve sustainability by optimizing both energy consumption and queuing delays. With fast inference times, the ability to generalize across diverse task types, and minimal data requirements for training, our approach offers a practical solution for data center scheduling. This framework demonstrates strong potential for advancing sustainability objectives in AI-driven infrastructure. Through our collaboration with a data center, we achieved a 32% reduction in energy consumption and a 30% decrease in waiting time.
Data centers in the cloud use a lot of energy and produce considerable carbon emissions because of the growing requirements for computational and AI-heavy processes. Currently, available cloud schedulers pay attention to performance and resource usage optimization without much consideration of carbon footprint efficiency, security compliance, and region-based regulations for cloud data centers. In this research paper, we suggest creating a Secure Carbon-Aware Scheduler that includes machine learning-based carbon intensity prediction, prioritization of the workload, and compliance-based scheduling for distributed cloud environments. Our solution includes using a hybrid approach to carbon monitoring by integrating live carbon intensity data from one pilot region with predicted data based on machine learning models for distributed regions. We trained three ML algorithms such as Random Forest, Gradient Boosting, and Linear Regression for three years of data on carbon intensity, and the best models were chosen according to R2 score and MAE metric.The scheduler performs an assessment of potential regions through a multi-criteria objective scoring function that is based on carbon footprint, security compliance, workload prioritization, and system performance. The architecture employs security enforcement methods that support policies such as GDPR and HIPAA, as well as asymmetric cryptography-based security features for sensitive workloads. Experiments have been conducted by simulating cloud workloads from fourteen different regions and showed promising results in terms of workload prioritization and better carbon-aware scheduling when compared to conventional scheduling algorithms.
Shreenidhi Anant Bhat, Premalatha H. M.· 2026 4th International Confe...· 0 citations
The surge of GPU-intensive workloads in artificial intelligence (AI) data centers drives massive energy demands, leading to soaring costs and significant stress on local power distribution networks. Coordinating delay-tolerant workload scheduling with power grid conditions via precise workload prediction can mitigate these issues. However, a critical gap remains in conventional approaches, i.e., minimizing prediction error does not necessarily lead to minimized downstream operational loss. Hence, this paper proposes an end-to-end Predict-Then-Schedule (PTS) framework that integrates upstream workload prediction with downstream scheduling optimization. By leveraging differentiable convex optimization, the PTS framework maps input features directly to optimal scheduling and enables gradient-based training. Furthermore, to respect the data center's capacity, a workload over-shifted loss combining electricity cost with a penalty for load-shedding is introduced to evaluate scheduling quality. Experiments demonstrate that the proposed framework significantly reduces operational cost and enhances system security compared to the conventional two-stage baseline.
Siqi Yan, Jiebao Zhang, Xianhong Yao et al.· 0 citations
Resource-constrained edge devices require efficient and adaptive health monitoring to ensure reliable operation under dynamic workloads. This paper proposes an LLM-driven, context-aware framework that integrates real-time system metrics, historical data, and task-specific importance levels for anomaly detection and prediction. Specifically, the framework forecasts the semantic health state of the edge device five minutes ahead based on recent monitoring observations, enabling proactive intervention before critical operating conditions are reached. Multidimensional metrics, including CPU, memory, temperature, network load, and process information, are transformed into structured time-series representations and used as input to pre-trained machine learning models. A large language model (LLM) acts as an orchestration layer, dynamically selecting the most appropriate predictive model based on system context and resource constraints. This enables adaptive trade-offs between accuracy, latency, and computational cost. Experimental results on Raspberry Pi devices show that the proposed approach achieves comparable or improved performance while reducing resource usage compared to static methods.
Ioannis Tzitzios, A. Dimara, Georgiana Petridou et al.· Electronics· 0 citations
The rapid growth of artificial intelligence training and inference workloads has produced a corresponding rise in data center energy consumption and associated carbon emissions, raising concerns about the environmental sustainability of continued AI scaling. This paper proposes a Sustainable AI Computing framework that combines carbon-aware dynamic workload scheduling with neural network optimization to reduce the operational carbon footprint of AI systems without proportionally sacrificing model performance. The proposed scheduler performs both spatial carbon shifting, routing deferrable workloads to data center regions with lower real-time grid carbon intensity, and temporal carbon shifting, delaying flexible jobs to lower-carbon time windows within deadline constraints, while a complementary neural network optimization module applies adaptive pruning, quantization, mixed-precision computation, and early-exit inference to reduce per-job energy consumption. The framework was evaluated across a simulated multi-region data center testbed spanning five regions with heterogeneous grid carbon intensity profiles, using representative training and inference workloads including convolutional and transformer-based models. Experimental results show that the proposed hybrid carbon-aware scheduler and optimizer combination reduces relative carbon emissions to 44% of a static round-robin baseline, compared to 71% for spatial shifting alone and 68% for temporal shifting alone, while cluster GPU utilization improves from an average of 52% under baseline scheduling to 78% under the proposed scheduler. Accuracy-energy trade-off analysis identifies a Pareto-efficient operating region in which up to 55% energy reduction per inference is achievable with less than one percentage point of accuracy degradation, beyond which further compression yields diminishing accuracy returns. Ablation results confirm that scheduling and model optimization contribute complementary and largely additive carbon reductions, with the combined framework outperforming either component in isolation. These findings demonstrate that meaningful reductions in AI-related carbon emissions are achievable through coordinated system-level and model-level interventions without requiring fundamental changes to underlying hardware infrastructure.
Rooban Agrawal, Sudeshna, Reeva Sharma et al.· International journal of com...· 0 citations
PRISM, a prediction-guided runtime framework that jointly selects model variants and CPU allocations for containerized edge microservices, and adapts each pipeline stage in place and minimizes predicted CPU-package energy under deadline, resource, and offline model-level Quality of Result constraints is presented.
Uwe Gropengießer, Thomas Reuter, Dominik Schön et al.· 0 citations
CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.
Zejian Wang, Nan Lin, Zinuo Cai et al.· ACM Transactions on Architec...· 0 citations