Aug 2026· Proceedings of the ACM SIGCOMM 2026 Conference· 0 citations· 30 references
TL;DR
The experience in designing, deploying, and operating Pegasus, a data center network tailored for the AI cloud, along with operational lessons learned from its deployment are shared.
Abstract
Today, AI cloud is key to serving diverse users with AI services, where cloud networking forms the basis. In this paper, we share our experience in designing, deploying, and operating Pegasus, a data center network tailored for the AI cloud, along with operational lessons learned from its deployment. The key designs of Pegasus include: 1) Network virtualization: a DPU-RNIC decoupled collaborative hardware architecture to enable a single DPU to virtualize multiple RNICs while reducing the power consumption. We design two-level flow tables on both DPU and RNICs to support underlay-overlay IP address translation and ensure isolation. For DPU-RNIC communication, we introduce a per-RNIC communication state machine to reduce communication overhead. 2) Network transport: customized and transparent transport offloading in the RNIC for low-latency and high-throughput communication performance for various AI workloads. We carefully offload per-packet load balancing and credit-based congestion control in RNICs, optimizing reorder delay and eliminating the impacts of hardware jitter. Pegasus has been deployed in production for over two years, currently covering 8K GPUs and supporting a wide range of tenants' AI applications.
Compute and memory resources in cloud environments are strictly managed and isolated by the control plane; in contrast, network resources lack equivalent management and isolation mechanisms. This best-effort treatment of networking leads to significant challenges for modern AI workloads, which have diverse and bandwidth-intensive communication patterns. Without fine-grained network resource control, these workloads suffer from interference, unpredictable throughput, and suboptimal cluster utilization. To address these issues, this paper demonstrates how network bandwidth can be elevated to a first-class, schedulable, and enforceable resource within Kubernetes, the de facto standard for cloud-native orchestration. We introduce a new scheduling capability that models network interfaces as allocatable resources and regulates bandwidth sharing through the Dynamic Resource Allocation (DRA) framework, with enforcement implemented using the Hierarchical Token Bucket (HTB) mechanism. We evaluate the system using multitenant AI workloads derived from real-world communication characteristics with a simulation-based approach and validate the proposed enforcement strategy in a real cluster. Results show that the proposed two-level bandwidth allocation improves tenant performance predictability and satisfaction while maintaining packed cluster utilization.
An architecture that decouples complex policy enforcement from high-speed packet forwarding to support VPC semantics on back-end NICs and enable front-end/back-end integration is proposed, suggesting that commodity hardware can support both high-throughput AI training and flexible VPC features.
Yinhe Wang, Xing Li, Enge Song et al.· Asia-Pacific Workshop on Net...· 0 citations
Optimizing burst-heavy datacenter workloads necessitates finegrained network control and visibility. We introduce CSIG, a protocol that delivers precise, multi-bit bottleneck congestion signals via a fixed-length Ethernet header. The architecture captures μsgranularity switch metrics, such as available bandwidth, and signals them to end-hosts using in-band, line-rate operations. We propose Fast Ramp-Up, a congestion control primitive that leverages these bottleneck signals to reduce median RPC latency by 20% and unclaimed bandwidth by 60% in production. Beyond transport-level performance, CSIG enables flow-aware observability by embedding μs-scale metrics into every packet, allowing individual application transfers to pinpoint their bottleneck location, such as the topology tier limiting their performance. CSIG thus transforms network telemetry from post-hoc correlation into a real time, context-aware capability. We demonstrate CSIG's broad deployability by validating it across five generations of commodity switch hardware (up to 102.4 Tbps), four NIC generations, and five transport stacks. Our design proves that a streamlined Layer 2 approach, focusing exclusively on the principal path bottleneck, provides transport-agnostic gains without requiring forklift hardware upgrades.
Abhiram Ravi, N. Dukkipati, Weiwu Pang et al.· 0 citations
Spillway introduces a DPU-host hybrid data plane that repurposes idle host CPU resources to process spillover traffic when the DPU becomes the bottleneck, and decouples virtual switching capacity from static DPU hardware limits.
Xiaochong Jiang, Dian Fan, Yilong Lv et al.· Proceedings of the ACM SIGCO...· 0 citations
AI training and inference are driving cloud networks toward terabit-per-second (Tbps) bandwidth per server, challenging the scalability and efficiency of today's cloud network architectures. A prevalent design scales bandwidth by stacking monolithic Data Processing Units (DPUs), but this approach tightly couples control and data plane resources, leading to excessive cost, power consumption, and operational complexity. We identify a fundamental control-data plane divergence in AI clouds: while data plane bandwidth demand grows rapidly, control plane demand remains largely flat due to the dominance of elephant flows. As a result, monolithic DPUs become systematically over-provisioned when used as bandwidth scaling primitives. We present DistDPU, a disaggregated DPU architecture that redefines the scaling abstraction for high-bandwidth cloud networking. DistDPU decomposes a monolithic DPU into lightweight, bandwidth-provisioning Execution Modules (EMs) and a shared, control-centric Orchestration Module (OM), enabling independent scaling of data and control plane resources. By scaling out low-cost EMs under a single OM, DistDPU exposes a unified, high-bandwidth logical DPU interface to the cloud management plane. To preserve RDMA performance and multi-tenant isolation at scale, we co-design the EM-OM functions and the inter-module fabric to minimize virtualization overhead while enforcing security and manageability invariants equivalent to those of a monolithic DPU. DistDPU has been deployed in production for two years. It serves more than 10,000 GPUs and delivers higher efficiency and strong performance on real-world AI workloads than state-of-the-art designs.
Hao Mei, Lizhou Gao, Yuanyi Zhu et al.· 0 citations
XFir is the first hardware-accelerated new-flow setup system for cloud host servers that delivers high CPS throughput while preserving sufficient flexibility, and is cost-effective to deploy, requiring only a single DPU per host.
Shihan Lin, Shunqiao Jiang, Liang Wang et al.· Proceedings of the ACM SIGCO...· 1 citation