Skip to content

EMA: Elastic and Performance Transparent Memory Across GPUs

Sep 2026 · arXiv.org · Vol abs/2609.27040 · 0 citations · 55 references
Computer Science

TL;DR

This work presents EMA, a memory sharing system that allows GPUs within a server to borrow and reclaim memory from each other, forming an elastic pool of capacity, and ensures performance transparency for both borrowers and lenders.

Abstract

Multi-GPU servers have become the standard building block of modern data centers, providing aggregated capacity through high-bandwidth interconnects. At the same time, workloads such as LLM inference exhibit highly dynamic memory demands, which can cause one GPU to exhaust its local memory while others remain underutilized. This mismatch motivates a model of elastic resource sharing across GPUs. We present EMA, a memory sharing system that allows GPUs within a server to borrow and reclaim memory from each other, forming an elastic pool of capacity. EMA ensures performance transparency for both borrowers and lenders. For borrowers, prefetching hides remote access costs so that applications experience remote and local memory as indistinguishable in performance. For lenders, borrowed resources remain reclaimable on demand, guaranteeing that performance never falls below that of static partitioning. While our design focuses on memory, the same principle naturally extends to other GPU resources. Our evaluation shows that EMA improves individual user throughput by up to 52%, achieves 96% of the throughput of a system provisioned with 2X capacity, and maintains latency similar to the static local baseline.

View source

Similar papers

Breaking the Tradeoff: Elastic and Isolated GPU Sharing with Ghost

Ghost is an OS-level GPU virtualization layer integrated directly into the open-source GPU driver, using a GPU container abstraction with cgroup -like APIs for compute and memory control and privileged hardware-level scheduling and preemption for dynamic compute resource management.

Unknown authors · 0 citations
Open access Oct 2026

Unified CPU–GPU Memory Performance Across Edge, Consumer, and Datacenter Platforms: Three Case Studies

Unified memory allows a central processing unit (CPU) and a graphics processing unit (GPU) to share data, simplifying the development of applications that use both processors. However, a shared memory interface does not eliminate the performance effects of where data reside, how often they are reused, or how the proces...

Mohammad Dashti · 0 citations
Book Open access 2027

Breaking the Tradeoff: Elastic and Isolated GPU Sharing with GVM

GPUs are expensive resources but often underutilized. This inefficiency is exacerbated by the increasing diversity of AI workloads, where small or specialized models often fail to saturate high-capacity devices, necessitating GPU sharing. Existing sharing approaches force a hard tradeoff. They either partition hardware...

Yi-Cheng Liu, Yi-Fan Qiao, Tian Xia et al. · 0 citations
Sep 2026

SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs

SlideDP is presented, a synchronous data-parallel runtime for shared-host multi-GPU systems that maintains one authoritative host state, decouples communication routes from state layout, and pipelines parameter delivery, gradient aggregation, and CPU updates across ranks and chunks.

Rui-Jia Yang, Shi-Yuan Lin, Yu-Long Ao et al. · 1 citation
Book Open access Sep 2026

Analysis of shared memory between CPUs and GPUs

This work investigates the memory capabilities of the NVIDIA DGX Spark, a novel platform featuring a unified memory architecture where DDR memory is located on the CPU and is fully accessible from the GPU.

Silvia R. Alcaraz, S. Hepkema, Vasilis Mageirakos et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.