RDMA vs. SSD Swap: A Preliminary Study through the Lens of Application Performance

Abstract

Memory disaggregation enables applications to extend local memory capacity using remote memory servers. A common approach implements transparent “far memory” using Linux swap over RDMA-capable InfiniBand networks, allowing pages to be swapped at microsecond-scale latency. This paper compares RDMA-backed swap with conventional Linux swap backed by local NVMe SSDs. We evaluate RDMA- and SSD-backed swap across multiple applications and memory pressure regimes. We find that RDMA-backed swap provides clear benefits when applications frequently access swapped-out pages. However, this is also the regime in which transparent memory offloading severely degrades application performance and violates SLOs that typically tolerate only a modest slowdown compared to local-memory execution. Maintaining such SLOs requires low swap traffic, under which the performance gap between RDMA- and SSD-backed swap narrows substantially. Our results suggest that modern NVMe SSDs, especially given their lower cost, constitute a surprisingly competitive baseline for transparent memory offloading under performance-preserving operating conditions. More broadly, we argue that comparisons between memory offloading systems should focus on memory pressure ranges where application performance experiences only modest degradation.