The results show that token attribution is not a reliable proxy for marginal energy under batched execution, and that measured Shapley ground truth can calibrate low-cost request features toward fairer attribution.
Abstract
Batched LLM serving improves throughput but complicates energy accounting. GPU power telemetry is aggregate, whereas sustainability reporting, chargeback, and workload analysis often require request-level energy charges. Existing inference-energy benchmarks report model-, phase-, or token-level energy, and recent carbon-accounting work motivates Shapley fairness conceptually. Neither provides measured request-level ground truth, so how far the accounting rules used in practice deviate from a fair allocation has remained unknown. We present JouleShare, an attribution framework with two components. An offline harness establishes this ground truth by replaying request subsets under vLLM with a reproducible protocol, integrating GPU power telemetry, and computing exact Shapley energy for each request. A lightweight calibration model, JCalib, then learns to predict Shapley shares from cheap request features for use at serving time. Across 16 model/workload runs, token-proportional attribution differs from exact Shapley by 0.440 normalized L1 on average under static batching and by 0.458 under continuous batching, a gap that reproduces across three data-center GPUs. JCalib reduces this error to 0.116 under static batching and 0.177 under continuous batching, below even a standalone-measurement baseline that is unavailable online, while preserving exact batch-energy efficiency. Sampled Shapley extends the measured reference to larger group sizes, where the gap persists and a single offline calibration remains the most accurate deployable rule. The results show that token attribution is not a reliable proxy for marginal energy under batched execution, and that measured Shapley ground truth can calibrate low-cost request features toward fairer attribution.
KV (Key-Value) volume is introduced, a physically grounded metric that captures the spatiotemporal footprint of a request’s KV cache occupancy, and it is shown that energy per KV volume (EPV) provides a stable and reproducible signature for modeling serving energy.
Xianyi Yuan, Hanlong Liao, Kunming Zhang et al.· Asia-Pacific Workshop on Net...· 0 citations
GPU energy consumption represents a primary operational cost for Large Language Model (LLM) inference services. Despite the inherent variability of production workloads, most deployments rely on static power configurations that fail to exploit the non-linear relationship between power limits and performance. Consequently, opportunities to reduce energy overhead while maintaining Service Level Agreements (SLAs) are often missed. While modern GPUs provide hardware-level powercapping interfaces, their systematic application in a feedbackdriven manner to satisfy stringent latency guarantees remains significantly underexplored. This paper proposes a lightweight, non-intrusive adaptive GPU power-capping mechanism based on a PID controller that dynamically adjusts power limits using SLA feedback. The controller operates directly through vendor power-limit interfaces and incorporates guardrails such as deadbands, rate limiting, utilization-aware gating, and antiwindup protection. We evaluate the approach on a server with eight NVIDIA H200 GPUs running the qwen3:32b model under a time-varying Poisson workload, with SLA compliance defined over TTFT. Results show that static caps expose a rigid energy-SLA trade-off, whereas the adaptive controller achieves a 3.84% reduction in total energy consumption relative to a 700 W baseline while maintaining empirical SLA non-inferiority. These findings demonstrate that feedback-driven GPU power modulation can improve energy efficiency without compromising latency stability.
Alex F. R. Trajano, C. Costa, Francisco V. J. Nobre et al.· Annual International Compute...· 0 citations
CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.
Zejian Wang, Nan Lin, Zinuo Cai et al.· ACM Transactions on Architec...· 0 citations
The increasing use of renewable energy in data centers creates an opportunity to reduce the carbon footprint of energy-intensive LLM inference workloads. Unlike traditional stable power supply, renewable generation fluctuates over time, making it difficult to match computation demand with available energy. However, existing LLM serving systems primarily optimize latency and throughput without considering energy supply dynamics, leading to underutilization of renewable energy and unnecessary reliance on thermal power, and consequently, higher carbon emissions. We present GreenAlign, a renewable-aware scheduling framework that addresses this mismatch by treating best-effort (BE) requests as temporally shiftable load. GreenAlign enforces a power-constrained policy that executes BE requests using only residual renewable energy under normal conditions, and introduces a backlog risk metric to selectively relax this constraint when deadline violations are imminent. To ensure responsiveness, it maintains standby capacity to absorb unpredictable latencycritical (LC) bursts and uses lightweight length estimation to handle request uncertainty. Simulation results show that GreenAlign significantly reduces thermal energy usage while preserving LC latency and BE deadline satisfaction.
Chang Liu, Jiacheng Liu, Xiaofeng Hou et al.· Fall Joint Computer Conferen...· 0 citations
Deploying large language models (LLMs) on edge nodes enables low-latency and privacy-preserving inference, but faces severe resource constraints under high-concurrence workloads. While existing inference systems leverage intranode key–value (KV) caching to improve efficiency, they largely neglect the unique complexities of multinode edge environments. Specifically, reactive KV cache eviction policies suffer from temporal uncertainty, often discarding reusable KV caches prematurely, while the tight coupling between request scheduling and cache placement often leads to myopic decisions that exacerbate load imbalance and resource contention. To address these challenges, we propose a dynamic block-level paradigm that treats KV blocks as the fundamental units for caching and scheduling, enabling dynamic sharing, generation, and eviction of arbitrary-length prefixes. We present complete modeling of the spatiotemporal coupling between scheduling and caching under block-level granularity, capturing intricate interactions overlooked by prior work. Based on this model, we design an online joint optimization algorithm, which applies to general edge LLM serving scenarios. The algorithm decouples spatiotemporal dependencies via randomized rounding over per-slot subproblems, achieving a balance between real-time responsiveness and long-term system efficiency. Theoretical analysis establishes high-probability near-optimality guarantees, and extensive experiments show that our method reduces the average time to first token (TTFT) by up to 54.02% over existing baselines.
Xishuo Li, Wei Jiao, Jun He et al.· IEEE Internet of Things Jour...· 0 citations
This paper studies a resource-allocation inefficiency in batched large language model (LLM) serving: heterogeneous requests that share a decode batch impose max-driven computational costs on one another. Because the wall-clock cost of a batch step is largely governed by the largest active KV-cache footprint, a short request co-batched with a long request can experience latency and GPU-resource consumption disproportionate to its own token workload. We formalize this phenomenon as a resource-fair scheduling problem. We develop a mathematical scheduling model that connects within-batch resource fairness to system throughput. The proposed fairness constraint bounds the disparity in decode progress, equivalently KV-cache footprint, among co-batched requests. Based on this model, we design the Insert-Short-Jobs-with-Limit (ISJL) algorithm, a parameterized hybrid batching policy. We prove that ISJL achieves a global competitive-ratio lower bound of $3/4$. We further examine the profit implications of resource-fair scheduling under the token-metered pricing convention used by commercial LLM APIs. Numerical experiments show that ISJL occupies a favorable middle ground between FCFS, which has large batching externalities, and LJF, which is cost-aligned but sacrifices batching flexibility. Thus, ISJL provides a bi-criterion scheduling policy: it maintains high throughput while aligning max-driven batch cost with token-metered revenue.