PIN: Less Is More for RDMA Load Balancing

Abstract

As RDMA becomes increasingly tolerant to out-of-order delivery, fine-grained packet-level multipathing is emerging as a practical design for datacenter fabrics. Packet spraying and related schemes improve load distribution for large flows, but they also force RDMA flows onto multiple paths whose conditions can differ at short timescales due to randomized traffic placement. For short flows, even one packet sent on a temporarily slower path can delay the entire flow. As a result, fine-grained load balancing can hurt, rather than help, small multi-packet flows. We present PIN, a lightweight wrapper that layers on top of any existing fine-grained RDMA load balancer. PIN applies less multipathing to short flows, which benefit from it least: it pins short flows to one path for their lifetime and leaves larger flows to the baseline mechanism. The design is simple, compositional, and requires only a size threshold. We show analytically that practical thresholds preserve overall load-balance fairness while reducing short-flow exposure to temporarily slower paths. Large-scale simulations across a variety of workloads show that PIN consistently improves both mean and tail completion time for short flows across multiple RDMA load-balancing baselines.