SwCC-Sec: Secure Software-Programmable and Per-Packet Congestion Control in RDMA Engine

Abstract

Many data centers adopt Remote Direct Memory Access (RDMA) to allow data center applications to achieve low latency and high throughput, while keeping minimal CPU overhead. The upper-layer applications keep evolving rapidly, and thus need congestion control algorithms (CCAs) that exist in the NIC hardware also to react correctly and timely, especially for a burstier ML workload. Even worse, the data center network will increase the line rate to 400 Gbps, even 800 Gbps soon. Therefore, how to reduce control loop delay for various CCAs becomes crucial to the performance of various applications. However, RDMA’s hardwired CCA is not able to satisfy such a requirement. To this end, we design and implement SwCC-Sec, an RDMA engine with on-NIC RISC-V cores that allows software-programmable and per-packet congestion control. To avoid the performance degradation caused by introducing the programmable RISC-V cores, SwCC-Sec carefully designs the 1) RISC-V core memory subsystem, 2) engine architecture, and 3) interaction between the RISC-V core and other NIC resources, and 4) an on-NIC hardware component to defend congestion-based attacks without sacrificing performance. Besides, SwCC-Sec provides a set of rich software APIs, allowing developers to deploy new CCAs with minimum engineering efforts. We prototype SwCC-Sec using the Xilinx U280 FPGA. Experimental results demonstrate that SwCC-Sec achieves performance comparable to current commercial RDMA NICs (Mellanox ConnectX-5). Both SwCC-Sec and ConnectX-5 reach $3.1~\mu $ s control loop RTT and need 512B packet size to reach line-rate traffic (100 Gbps). In terms of flexibility, SwCC-Sec allows to use the C language to implement nearly all kinds of existing CCAs. The potential ASIC design of SwCC-Sec can easily scale to higher network bandwidth.