A method for scaling Kubernetes controllers horizontally that combines lightweight load balancing with cache synchronization with cache synchronization is proposed and implemented and evaluated.
Abstract
As Kubernetes clusters grow, the scalability of controllers can become a bottleneck for the performance of the system. Distributing the load dynamically across multiple controller instances, however, raises the following two problems, and a controller can therefore be run only as a single instance today. The first problem is the cost of reassignment. A controller retrieves objects on the basis of the Labels attached to them, so in a naive design in which the assigned instance is recorded in a Label on every object, the Labels must be rewritten in proportion to the total number of objects whenever instances are added or removed. The second problem is cache consistency. A controller consults only its own cache when it reads an object and never refers to the actual data, so the cache has to be updated explicitly at the time of a reassignment. Furthermore, a controller manages a cache independently for each kind of object, so cache updates have to be synchronized across the kinds of objects. We propose a method for scaling Kubernetes controllers horizontally that combines lightweight load balancing with cache synchronization. A two-level Hash maps objects to Virtual Nodes, records their identifiers in Labels, and assigns Virtual Nodes to instances by Consistent Hashing. Whenever instances are added or removed, it therefore suffices to update the Label value specified when objects are retrieved, and no Label on an object has to be rewritten. The cache is also locked until the reassignment has completed, which prevents any reference to a stale cache. The version identifier of the data store is used to synchronize the kinds of objects with one another. We implemented the proposed method on Kubernetes and evaluated it: the processing throughput rises with the number of instances, and the time required for reassignment remains within an acceptable range.
The default scheduler of Kubernetes, the state-of-the-art container orchestrator, uses fast, local placement decisions. Unfortunately, this design leads to resource fragmentation, reduced cluster usage, and overprovisioning. External solvers can compute global placement plans, but enforcing these plans in upstream clusters is hard. Kubernetes provides no native cross-node preemption, uncoordinated concurrent scheduling leads to inconsistencies, and replacing the default scheduler would sever deployments from upstream cycles. We present OPSche, an open-source Kubernetes Scheduling Framework plugin where external solvers can drive cluster-wide placement decisions in concert with the default scheduler. OPSche atomically validates and enforces solver-produced plans through coordinated framework hooks and supports three trigger modes: scheduling-failure, periodic, and stable-queue -- resp. triggered when a workload cannot be placed, at fixed time intervals, when the set of pending workloads stabilises. Each mode has a blocking variant for a finer tuning of placement quality, latency, and disruption. We pair OPSche with a constraint-based optimisation solver, showing its feasibility across a broad set of cluster configurations and reporting improvements of resource usage by up to 3.0% and scheduling latency by more than a second.
Henrik Christensen, S. Giallorenzo, J. Mauro· 0 citations
A queueing network (QN) model of the rebalance process of the process used in MongoDB is developed, and a preliminary evaluation of performance prediction by the QN model via simulations is described, to understand the impact of different parameters of the rebalance process.
G. Paterakis, E. Ntallaris, K. Magoutis et al.· International Symposium on C...· 0 citations
A comprehensive review of Kubernetes scheduling strategies published between January 2023 and January 2026 is presented and a multi-dimensional taxonomy is established that categorizes scheduling approaches based on common objectives, modification methods, optimization methodologies, targeted workloads, evaluation methods, scheduling scopes, and performance metrics.
Mohammed Alhakimi, R. Latip· Computers· 0 citations
When affinity recovers too little KV work, its residual load skew reduces or erases the improvement, so gating any deployment with a shadow replay rather than enabling affinity from workload statistics alone is recommended.
In this paper, we introduce KDB, a novel persistent key-value data store (a concurrent index) with rich linearizable semantics. In contrast to state-of-the-art systems which offer only lookup and put/remove operations, KDB supports both snapshots (which are used by range scans) and atomic batch updates—put and remove operations that are executed atomically. Despite its rich semantics, our system offers highly scalable performance across varied workloads thanks to its unique multiversioned architecture. It features a hybrid lock-CAS synchronization mechanism that allows lookup operations and scans to proceed in a wait-free fashion. Under the hood, KDB maintains all key-value entries in persistent memory (PM) for failure atomicity, but it heavily relies on an efficient DRAM-backed multiversion index based on skip lists to hide the costs of accessing PM. For better PM utilization, entries are arranged in PM in preallocated arrays that occasionally undergo compaction.
Tadeusz Kobus, Maciej Kokociński, Krzysztof Kortas et al.· ACM Symposium on Parallelism...· 0 citations
Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack, including model layout and loading, expert residency, CPU--GPU execution, agentic state reuse, and runtime memory management, around two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine. Rather than committing to a fixed offloading strategy, FreeToken continuously maps computation and model state onto the resources actually available. FreeToken supports more than 20 MoE models and real coding and tool-using agents across hardware ranging from an 8GB laptop GPU to a single workstation GPU. More importantly, it changes what these machines can practically serve, from a 35B model on a laptop to a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. FreeToken turns open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence. We release the system at flashml.ai.
Shuo Yang, Xiao-yun Fan, Melissa Z. Pan et al.· 2 citations