PRISM: Predictive Runtime In-place Scaling and Model Selection for Edge Microservices

Uncertainty-Aware Resource Allocation for Multi-Path Programs with In-Kernel Predictions

CELLServe: An SLO-Aware and Cost Efficient LLMs Serving System for Serverless Computing Environments

CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.

Estimating Execution Progress of Black-Box Tasks with Resource-Consumption Models

In cloud environments, recurring workloads such as data processing jobs, CI/CD tasks and machine learning pipelines are typically executed as opaque black-box programs. Their internal progress is not directly observable, yet their resource-consumption patterns often exhibit structural similarity across runs despite variations in input data and hardware. Accurately estimating the progress of such running tasks can bring major benefits to scheduling and resource optimization. Using resource-consumption models of such tasks, derived from previous task execution monitoring, to accurately estimate a tasks cumulative progress allows detailed and accurate proactive resource steering, planning and scheduling. We present a novel approach that, given a generic resource-consumption model, estimates, based on current resource monitoring data, the progress of an ongoing execution in the form of a probability distribution which supports uncertainty quantification. The approach accounts for deviations from the model caused by varying system performance profiles, input sizes and parameters. It performs estimation via Markov-chain Monte-Carlo sampling around a maximum a posteriori estimate. Synthetic evaluations show that the approach reliably estimates task progress with an error of approximately 0.09 when suitable priors were selected. Real-world demonstrations show that the approach remains effective, and that performance degrades only slightly when the method is applied to real data. The degree of robustness to poor prior selection has also been thoroughly explored.

C-PCP: Correlation-Aware Proportional Critical Path Scheduling for Input-Sensitive Serverless Workflows

OrionInfer: Low-Overhead Parallelism Switching and Live Migration for Efficient LLM Serving

Or OrionInfer, an adaptive LLM serving system that aligns inference strategies with real-time demand and introduces three key techniques: runtime switching between data parallelism and tensor parallelism with negligible overhead, an efficient inference pipeline that preserves batching efficiency during parallelism transitions, and live-migration-based load balancing to alleviate memory pressure and improve resource utilization.

Energy-Efficient LLM Inference with SLA-Constrained Adaptive GPU Power Capping

GPU energy consumption represents a primary operational cost for Large Language Model (LLM) inference services. Despite the inherent variability of production workloads, most deployments rely on static power configurations that fail to exploit the non-linear relationship between power limits and performance. Consequently, opportunities to reduce energy overhead while maintaining Service Level Agreements (SLAs) are often missed. While modern GPUs provide hardware-level powercapping interfaces, their systematic application in a feedbackdriven manner to satisfy stringent latency guarantees remains significantly underexplored. This paper proposes a lightweight, non-intrusive adaptive GPU power-capping mechanism based on a PID controller that dynamically adjusts power limits using SLA feedback. The controller operates directly through vendor power-limit interfaces and incorporates guardrails such as deadbands, rate limiting, utilization-aware gating, and antiwindup protection. We evaluate the approach on a server with eight NVIDIA H200 GPUs running the qwen3:32b model under a time-varying Poisson workload, with SLA compliance defined over TTFT. Results show that static caps expose a rigid energy-SLA trade-off, whereas the adaptive controller achieves a 3.84% reduction in total energy consumption relative to a 700 W baseline while maintaining empirical SLA non-inferiority. These findings demonstrate that feedback-driven GPU power modulation can improve energy efficiency without compromising latency stability.