RedLB: Load Balancing With a Redundant QP for RoCE

Abstract

With the rapid development of network intensive applications such as LLM training and cloud storage, RoCE has been widely deployed in data center networks to enhance the underlying high-speed network and meet the high-performance requirements of upper layer applications. The existing host-based RoCE load balancing schemes typically utilize multiple QPs to achieve multi-path transmission. However, commercial RNICs fall short on scalability and only support a small number of high-performance QP connections. Therefore, they can only guarantee performance in specific scenarios. Meanwhile, commercial RNICs require in-order delivery of packets, which prevents arbitrary rerouting. In this paper, we propose RedLB, a host-based load balancing framework for RoCE. The key design of RedLB is to generate a redundant QP on the end-host side for implementing rerouting of low-overhead in-order delivery. Meanwhile, RedLB achieves accurate congestion information collection through message segmentation, guiding efficient rerouting. We have implemented RedLB on commercial RNICs. We conduct evaluations using both testbed experiments and NS-3 simulations. The extensive evaluation results show that RedLB can achieve up to 48.2% and 40.5% improvement for average and $99^{th}$ -percentile FCT, respectively compared to the state-of-the-art load balancing algorithms.