Aug 2026· Asia-Pacific Workshop on Networking· 0 citations· 38 references
Computer Science
TL;DR
SprayCast is proposed, a congestion-adaptive native RDMA multicast scheme for dynamic sparse token Dispatch that uses in-band network telemetry feedback to steer replication away from congested multicast branches and range-based negative acknowledgments for localized loss recovery, saving bandwidth and reducing tail latency in dynamic all-to-all communication.
Abstract
Mixture-of-Experts (MoE) models outperform traditional dense models through sparse expert activation, where each token is dynamically routed to a small subset of experts. Across many tokens, these sparse Dispatch operations induce all-to-all traffic, making communication a major bottleneck for both training and inference: unicast replication wastes bandwidth, while table-driven multicast struggles with receiver-set churn and incast. In this paper, we propose SprayCast, a congestion-adaptive native RDMA multicast scheme for dynamic sparse token Dispatch. To avoid maintaining multicast forwarding tables in switches, SprayCast encodes each packet’s destination node set in its packet header using hierarchical bitmaps, enabling table-free in-network replication. It uses in-band network telemetry (INT) feedback to steer replication away from congested multicast branches and range-based negative acknowledgments (NACKs) for localized loss recovery, saving bandwidth and reducing tail latency in dynamic all-to-all communication. In htsim simulations on a 128-server fat-tree, SprayCast achieves better scalability as top-K dispatch fanout increases and reduces P99 dispatch tail latency by up to 6 × at K = 8 compared with representative baselines.
LLM deployment is migrating from data centers to edge devices, where Mixture-of-Experts (MoE) models offer a promising path: sparse expert activation allows the model to be spread across multiple low-cost edge nodes. Distributed MoE inference repeatedly dispatches embeddings from one main node to many workers - a one-to-many pattern poorly served by the sequential unicasts of mainstream stacks (NCCL, TCP), yet naturally matched by UDP broadcast. We propose a UDP broadcast method for collaborative edge MoE inference, augmented with timeout-driven retransmission exploiting near deterministic latency in distributed MoE for reliability and unordered result gathering for robustness to expert mispredictions, yielding a consistent 1.4x speedup over NCCL and TCP on a wired 8-node cluster. In wireless settings, however, we uncover a deeper, long-forgotten bottleneck: IEEE 802.11 caps broadcast rates at 54 Mbps regardless of physical-layer capacity - a legacy policy built for sparse control traffic, not edge AI. NS-3 simulations at distances 1m, 2m and 5m show that the optimal rates are much higher (64x, 43x, and 32x, respectively) than the 54 Mbps cap applied in standard. Thus, we argue that broadcast is no longer a control-plane relic: it is time for Wi-Fi standards to treat it as a high-throughput data-plane citizen.
Liujianfu Wang, Yuyang Du, Shiqi Xu et al.· 0 citations
Modern CDN vendors typically provide multi-server services for content deliveries, in which different requests from the same client are often scheduled to diverse CDN nodes. In this case, the requested data (e.g., short video chunks) always suffer from cold start during its transmission, especially for the first flow of established connections, due to the unknown client-side network quality. Existing schemes focus on single-server startup parameter optimization, whose referred metrics appear to be extremely limited. To address this issue, we propose hStart, a novel collective service paradigm that enables hot-start congestion control by efficient collaboration between CDN nodes. hStart synchronizes probed network quality information (e.g., bandwidth, latency) across CDN nodes to initialize congestion control from historical data, avoiding probing convergence. It uses hierarchical state synchronization to limit overhead, and adaptive prediction with a time-decay trust model to handle dynamics. We implement and deploy the hStart system for the QUIC protocol. Extensive evaluations in both controlled testbeds and large-scale production CDN environments demonstrate that hStart effectively eliminates cold start penalties, boosting 46.47% throughput gain and 21.43% rebuffering reduction.
Tong Li, Jiuxiang Zhu, Bo Wu et al.· Asia-Pacific Workshop on Net...· 0 citations
With the rapid growth of large language model training and generative artificial intelligence services, data center networks face severe micro-burst traffic and high concurrency. Traditional hash-based flow-level load balancing cannot sense link states, leading to hash collisions, hotspot congestion, and tail latency in multipath Clos networks. Existing packet-level schemes are constrained by stale state information, high hardware complexity, and poor adaptation to heterogeneous links. To address these issues, this paper proposes probabilistic state-proportional (PSP) dispatching, a packet-level load balancing algorithm. Using a Band-based discrete state representation, PSP replaces global sorting with local probability mapping, reducing hardware complexity while suppressing herding and oscillations caused by stale states. Experiments on a cycle-accurate simulator show that PSP is robust across port scales, bandwidth-limited paths, and fixed-flow interference. It outperforms join-the-shortest-queue (JSQ) scheduling and Random in loss rate, 99th-percentile buffer occupancy, and scalability, while remaining competitive with Top-k at lower hardware cost. PSP provides an effective balance among performance, stability, and overhead for artificial intelligence data centers.
Jiaqi Liu, Chunyang Zhang, Heng Pan et al.· 0 citations
To achieve high-throughput and low-latency Remote Direct Memory Access (RDMA) communication in data center networks, load balancing is critical for preventing congestion and ensuring that traffic is efficiently distributed across available network paths. However, existing schemes may not effectively detect rerouting opportunities in continuous RDMA packet streams and may degrade in-order delivery, limiting their applicability to RDMA traffic. To address these limitations, we propose ProLet, a load balancing scheme that enables proactive probing and reroutes elephant flows at flowlet granularity in lossless RDMA networks. ProLet dynamically fine-tunes per-destination top-of-rack timeouts and enables effective in-network flowlet identification based on real-time network conditions. Meanwhile, it leverages lightweight mice flows as proactive probes to maintain network-wide congestion awareness. This allows ProLet to reroute elephant flows before congestion accumulates, mitigating the persistent queue buildup inherent in subflow-based schemes. Extensive numerical evaluations demonstrate that ProLet reduces average and tail flow completion time slowdowns by 69% and 79%, respectively, compared to state-of-the-art load balancing schemes.
Hong Wang, Jinhao Luo, J. Tan et al.· Asia-Pacific Workshop on Net...· 0 citations
Volumetric streaming remains difficult to scale because receivers with overlapping fields of view are often served independently, causing repeated transmission of shared content. We present MD2G-Cast, a relay-coordinated multicast framework over Media over QUIC with an application-aware control layer for scalable multi-user volumetric delivery. MD2G-Cast jointly uses viewing overlap, receiver capability, and bandwidth conditions to form reusable multicast groups, share common Base content, and selectively admit Enhanced delivery. We formulate grouping and Enhanced admission as a sequential control problem, realize it with Proximal Policy Optimization (PPO), and train a compact relay model with teacher guidance for Enhanced admission. We implement MD2G-Cast with real MoQ processes and evaluate it with real access and 6DoF viewing traces for up to 100 users. At 20 and 100 users, MD2G-Cast keeps the receiver-side $P_{99}$ delivery interval below 40 ms across all seven access profiles, while Rolling reaches the 500 ms reporting cap in most cases. Across the evaluated user scales, MD2G-Cast achieves the highest or tied-highest mean system utility under homogeneous access and the highest mean utility under heterogeneous access, while reducing aggregate link load by about 27% relative to Clustering at 100 users. A matched relay-control ablation separates the control structure from its optimizer, showing that random feasible actions reduce utility while deterministic control remains competitive with PPO. Together, the results support relay coordination and selective Enhanced admission, rather than a particular policy optimizer, as the central design contribution.
Ruonan Chai, Yisu Wang, Zili Meng et al.· 0 citations
Optically switched networks suit the regular communication of dense ML models, but MoE introduces sparse, runtime-dependent traffic. We show that efficient offline-optimized routing enables efficient MoE training and inference on direct-connect topologies without the need for MoE traffic matrix or dynamic topology reconfiguration. MoX constructs token-aware multicast trees to reduce bandwidth tax, then uses static, precomputed link weights to balance traffic by solving a restricted multicast tree-packing problem. Using recorded traffic from large MoE models, token-level traces, and ASTRA-sim, we find that MoX accelerates the full MoE block -- dispatch, expert computation, and combine -- by up to 1.8x over min-hop routing. Moreover, it attains nearly ideal packet-switched network performance in random expander topologies. On a 1,024-TPU model of Google's Boardfly topology, MoX reduces the dispatch bottleneck link load by up to 47%. These results show that high-performance MoE on static direct-connect fabrics can be achieved via optimized load-oblivious routing without demand-driven reconfiguration.
Ori Cohen, Jakob Krebs, Daniel Amir et al.· 0 citations
