Results show up to 2.3x lower remote-access overhead compared to TCP-based designs, a 13% increase in cluster-wide memory utilization at rack scale, and up to 63% reduction in miss rate for memory-constrained tenants under skewed workloads.
Abstract
To handle unpredictable workloads, cloud providers typically over-provision memory to meet peak demand, resulting in substantial underutilization across datacenter clusters. At the same time, memory-constrained tenants may suffer elevated cache miss rates, even when idle capacity remains stranded elsewhere in the infrastructure. MemExchange is a cluster-wide, multi-tenant memory management system that dynamically right-sizes in-memory caching tenants according to workload demand. Leveraging marginal-utility-based allocation derived from online Miss Ratio Curve (MRC) estimation, MemExchange redistributes idle memory between tenants across physical nodes using RDMA. This approach transforms the dedicated caching memory scattered across servers into a logically aggregated pool, enabling cross-node memory exchange without centralized coordination or forced tenant co-location. To support efficient remote access, we design the MemExchange Tracker Communication (MTC) protocol, an application-layer mechanism that coordinates memory reallocation and enables one-sided RDMA operations without involving remote CPUs. We implement MemExchange in Memcached and evaluate it through microbenchmarks, medium and rack-scale deployments of up to 100 CloudLab servers. Our results show up to 2.3x lower remote-access overhead compared to TCP-based designs, a 13% increase in cluster-wide memory utilization at rack scale, and up to 63% reduction in miss rate for memory-constrained tenants under skewed workloads.
Tiered memory environments that combine DRAM and CXL provide substantial scalability in modern systems, yet they introduce critical fairness challenges for containerized workloads. Existing Linux kernel memory management primarily relies on access-frequency-based page promotion and demotion to optimize system-wide performance. This approach causes two severe problems: (1) flaws in the current Cgroups memory-limit implementation can trigger infinite page reclamation loops in tiered memory systems, and (2) memory-intensive workloads monopolize fast-tier memory (e.g., DRAM), leading to the noisy neighbor problem in which tier occupancy becomes highly unfair even when containers have identical memory limits. This paper resolves the infinite page reclamation loop problem and proposes FATE, a fairness-aware memory allocation for multi-tenant tiered memory environments. FATE extends the Linux Cgroups memory controller to enforce per-tier high-watermark thresholds that are dynamically proportional to the share of total memory allocated to each container. For workloads requiring deterministic performance, FATE also supports configurable static fast-tier reservations. To enable precise enforcement, we introduce a tier-aware page counter that tracks per-NUMA-node, per-tier usage during allocation, migration, and reclamation. By strictly respecting fast-tier thresholds, FATE eliminates infinite reclamation loops and prevents unnecessary OOM killer invocation. Evaluation with diverse workloads in multi-container environments shows that FATE consistently achieves fair tiered memory allocation across containers with negligible runtime overhead. When occupancy ratios were quantified using Jain’s Fairness Index, FATE improved fairness by 7.3% to 31.7% over the baseline. More importantly, FATE significantly enhances performance predictability—a critical requirement for cloud service-level objectives—by providing strict isolation across memory tiers.
Minwoo Jo, Binwon Song, Hayong Jeong et al.· IEEE International Conferenc...· 0 citations
A queueing network (QN) model of the rebalance process of the process used in MongoDB is developed, and a preliminary evaluation of performance prediction by the QN model via simulations is described, to understand the impact of different parameters of the rebalance process.
G. Paterakis, E. Ntallaris, K. Magoutis et al.· International Symposium on C...· 0 citations
STORM is presented, a NIC-level scheduler for all types of RDMA workloads using NIC-only information: the known RDMA request size, and per-queue-pair backlog, and converts these signals into a small number of extra priority levels on the wire and prioritizes requests that are either near completion or blocking queued dependent work.
Jichun Wu, Ran Shu, Gianni Antichi et al.· Proceedings of the ACM SIGCO...· 0 citations
While CXL presents a promising hardware substrate for Distributed Shared Memory (DSM), seamlessly scaling multithreaded applications across multiple nodes remains a formidable challenge. Existing CXL-based DSMs fall short: they require manual code modifications to share non-heap data, employ rigid data placement policies that fail under diverse and dynamic workloads, and suffer from severe page-fault processing overheads in sub-microsecond ($\mu\mathrm{s}$) environments. We present xDSM, a full-space, elastic DSM system built over CXL that transparently scales unmodified multithreaded applications. To eliminate the burden of manual code rewrites, xDSM employs an OS-runtime co-design that establishes a globally coordinated address space, seamlessly sharing all memory segments. To mask CXL access penalties, xDSM abandons static placement rules in favor of a dynamic, latency-driven policy that actively balances data between local DRAM and CXL memory. Finally, to resolve the fundamental tension between high base-page fault overheads and severe huge-page false sharing, xDSM introduces spatial locality-aware elasticity, dynamically coalescing and splitting pages on the fly to amortize processing costs. Evaluated across diverse workloads using 15 system configurations, xDSM outperforms CXL-only baselines by 1.5$\times$ to 2.2$\times$ and state-of-the-art hybrid DSMs by 1.1$\times$ to 2.2$\times$, while achieving near-linear scalability.
Guowei Liu, Kang Chen, Laiping Zhao et al.· 0 citations
This work presents Oneiros, a dynamic remapping engine for multi-tenant LLM serving that dynamically repurposes GPU memory allocated for model parameters as KV cache capacity, enabling nonblocking, unidirectional parameter transfer.