Jul 2026· IEEE Transactions on Services Computing· Vol 19, pp. 2716-2729· 0 citations· 46 references
Abstract
Complex microservices topology poses significant challenges to resource scheduling. Consequently, most schedulers rely on extensive offline analysis to prevent potential SLO violations resulting from inaccurate online decisions, leading to high deployment costs. Furthermore, regarding metric selection that is important to bridge the scheduling decision to actual resource allocation, although CPU throttling-based auto-scaling has emerged as a promising approach to capture latency sensitivity, existing methods lack a quantitative mechanism to map throttling signals to resource quota, relying instead on coarse heuristics that potentially waste resources. To address these issues, we propose Bayesian Optimization and Throttling based Vertical Pod Autoscaling (BOTVPA), a sample-efficient scheduling method. BOTVPA features a two-tier architecture: 1) Central Scheduler: We employ Bayesian Optimization (BO) to enable rapid decision-making with minimal samples, eliminating offline training overhead. 2) Local Scheduler: We introduce a novel CPU throttling model that mathematically bridges the gap between throttling rates and CPU allocation. Leveraging this model, the local scheduler dynamically translates BO’s decision into precise CPU resource limits at runtime. Results show that BOTVPA reduces CPU consumption by up to 19.4% over the best-performing baseline while satisfying SLO.
The results show that soft SLO limits reduce corrective rescheduling actions by 49% compared to hard-limit approaches while maintaining acceptable performance guarantees, and resource-aware scheduling decreases node-level congestion and further mitigates SLO violations, demonstrating the effectiveness of incorporating application-level flexibility and hardware-level insights into scheduling and rescheduling decisions.
Oliver Larsson, Thijs Metsch, Cristian Klein et al.· 0 citations
While spot instances offer a cost-effective alternative to on-demand cloud resources, they introduce reliability challenges for latency-sensitive microservices due to preemption risks and unpredictable provisioning delays. Conventional resource management systems, which often rely on assumptions of immediate instance availability, fail to account for these operational realities—resulting in increased risk of SLO violations when deployed in spot-based environments. In this paper, we propose Cremes, an adaptive and cost-efficient scaling framework that ensures microservice recovery within the spot instance grace period. Cremes explicitly models both instance waiting time and microservice startup latency, leverages cloud-exposed availability metrics, and applies lightweight machine learning for end-to-end latency prediction. By integrating these components into a multi-dimensional optimization engine, Cremes minimizes cost while satisfying recovery and performance constraints. Evaluations on AWS instances using DeathStarBench, TrainTicket, and Alibaba trace-driven experiments show that Cremes reduces infrastructure cost by up to 37.1% and maintains SLO violation rates under preemptible environments below 6.7%.
Liao Chen, Chenyu Lin, Junlin Chen et al.· IEEE International Symposium...· 0 citations
A hybrid congestion-aware autoscaling framework for serverless platforms inspired by TCP slow-start and congestion control principles is proposed, which significantly improves latency performance, scalability, and resource efficiency in serverless environments.
Mohammad Tari, Mostafa Ghobayee-Arani, Jafar Pouramini· International Journal of Clo...· 0 citations
A comprehensive review of Kubernetes scheduling strategies published between January 2023 and January 2026 is presented and a multi-dimensional taxonomy is established that categorizes scheduling approaches based on common objectives, modification methods, optimization methodologies, targeted workloads, evaluation methods, scheduling scopes, and performance metrics.
Mohammed Alhakimi, R. Latip· Computers· 0 citations
CELLServe formalizes SLO-constrained joint resource provisioning as an optimization problem with a dedicated algorithm, and introduces an opportunistic instance merging strategy for decode phase functions to reclaim fragmented resources.
Zejian Wang, Nan Lin, Zinuo Cai et al.· ACM Transactions on Architec...· 0 citations
Function-as-a-Service offers cost efficiency but often suffers from resource underutilization. This underutilization stems from the instance-level resource provisioning pattern, an issue that existing optimizations have failed to resolve fundamentally. The core problem is that static coarse-grained instance-level resource allocation cannot match the millisecond-level burstiness of dynamic requests. Consequently, it is difficult for current systems to achieve high resource utilization while maintaining high quality of service (QoS) guarantees. To address the problem, this paper advocates a shift to request-level resource provisioning, which redefines the individual request as the atomic unit for scheduling and resource management. We implement this approach in RRP, a scalable FaaS platform that enables efficient per-request resource allocation and release. RRP unifies instance placement and request routing with low-overhead, millisecond-level global visibility. Our evaluation shows that RRP significantly outperforms state-of-the-art instance-level platforms and algorithms. By matching resources to each request’s needs and isolating them from contention, RRP achieves low latency and high utilization. Specifically, on real-world Azure traces, RRP achieves speedups of 1.33 × –30.15 × for average end-to-end latency and 1.37 × –61.46 × for P99 latency, and raises CPU utilization from 44.80%–56.32% to 72.49% under bursty loads.
Runfu Li, Zishu Yu, Yifan Wang et al.· IEEE International Symposium...· 0 citations