TL;DR
CAPA is presented, a confidence-aware predictive autoscaler that couples a lightweight rolling ridge forecaster with an online one-sided residual-quantile bound and an SLO-debt feedback signal and reduces the estimated SLO violation rate relative to a reactive HPA-style baseline.
Large language model (LLM) inference deployed on Kubernetes faces an autoscaling challenge that conventional web services do not: new serving replicas take two to ten minutes to start because multi-gigabyte model weights must be loaded, which makes purely reactive scaling structurally late. We ask a sharp question: amo...
Cloud data platforms increasingly co-locate hybrid analytical workloads (data-lake/warehouse queries, batch and stream engines) with generative-AI (GenAI) inference services, producing load traces that are strongly bursty, heavy-tailed, and multi-phase. Reactive threshold autoscalers and simple ARIMA/LSTM predictors re...
Cloud-native machine learning inference services must hold tail-latency service level objectives (SLOs) while paying for as few replicas as possible. Two families of autoscalers compete for this job: reactive threshold controllers such as the Kubernetes horizontal pod autoscaler, and predictive controllers that provisi...
SLO-Scaler is proposed, an uncertainty-aware autoscaling framework that predicts short-horizon request rates, tail latency, and SLO violation probability using a Bayesian LSTM model, and is evaluated on the DeathStarBench Social Network benchmark deployed on Kubernetes under periodic, bursty, and long-tail traffic patt...
A reinforcement learning (RL) recommender VERA is presented that formulates vertical memory scaling as a Markov Decision Process and trains an agent on 3353 real Prometheus traces and demonstrates that an observation-driven RL recommender could outperform retrospective heuristics for dynamic memory scaling.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
