May 2026· Proceedings of the ACM SIGCOMM 2026 Conference· 0 citations· 30 references
Computer Science
TL;DR
ReTri, a bidirectional All-to-All schedule for ORNs based on the Trivance algorithm is presented, a bidirectional All-to-All schedule for ORNs based on the Trivance algorithm that improves completion time and improves reconfigurable Bruck by up to 2.1×.
Abstract
All-to-All collective communication is a key performance bottleneck for distributed machine learning (ML) and high-performance computing (HPC) workloads, where dense traffic increasingly stresses scale-up interconnects. While these ML and HPC workloads have driven unprecedented infrastructure demand, optical reconfigurable networks (ORNs) offer a promising path forward as they can reconfigure the network at runtime. By adapting the physical topology to the active workload, they improve communication cost and bandwidth utilization. However, optical reconfigurable networks introduce a fundamental trade-off for collective communication: each reconfiguration requires global synchronization, during which communication is suspended for at a non-negligible delay. Additionally, their benefit is critically contingent on whether the collective consists of structured phases that can be served by sparse and reusable topology states. In this paper, we revisit Bruck's All-to-All implementation and demonstrate the benefits of topology optimization in which both communication pattern and reconfiguration strategy are co-designed. We present ReTri, a bidirectional All-to-All schedule for ORNs based on the Trivance algorithm. ReTri uses balanced ternary block propagation to complete All-to-All in ⌈log3 n⌉ phases. The reconfiguration strategy induced by ReTri's pairwise bidirectional exchanges allows reconfiguration delays to be amortized across multiple phases. Preliminary simulations show that ReTri improves completion time by up to 10× over Pairwise All-to-All, even for millisecond-scale reconfiguration delays, and improves reconfigurable Bruck by up to 2.1×.
Modern collective communication libraries (CCLs) execute a collective communication task (CCT) by decomposing it into multiple sub-tasks, each mapped to a specific Virtual Topology (VT), which is an ordered graph of GPUs (e.g., a ring or a tree), to maximize parallelism and link utilization. As AI training scales to larger clusters, network anomalies (congestion and failures) are unavoidable, and a single straggling VT can delay the entire CCT. Existing solutions either rely on low-level transport-layer solutions which lacks a cross-sub-task perspective, or static CCL scheduling, failing to adapt to the dynamic and heterogeneous networks. We present PReCCL, a drop-in NCCL replacement that combines software inband telemetry with cross-VT workload reallocation. Unlike hardware-dependent telemetry (e.g., P4-INT), PReCCL implements in-band monitoring within the CCL. It precisely measures the stall counts of each VT, and piggybacks the telemetry meta-data on existing collective traffic. With this fine-grained visibility, PReCCL employs a dynamic cross-VT reallocation mechanism that shifts workloads from straggler VTs (suffering from congestion or faults) to healthy ones. Reallocation occurs only at CCT boundaries using an epoch-based deterministic protocol, avoiding cross-rank consistency races. We implement PReCCL in NCCL and evaluate it on a 32-GPU testbed, large-scale simulation, and a 1024-GPU production cluster. On the testbed, PReCCL improves CCT completion by up to 2.1X and end-to-end training by 1.21X, with low overhead above tuned message-size thresholds.
Zhiyong Chen, Kaihui Gao, Li Chen et al.· Proceedings of the ACM SIGCO...· 0 citations
Collective communication operations in AI and HPC workloads generate heavy network traffic. Offloading these operations to network switches reduces latency, but performing arithmetic and replication at line rate is difficult, especially as port counts and link speeds grow. Existing in-network approaches rely on accumulation buffers that not only limit throughput but also require complex state management to handle stragglers and congestion. We present CCSwitch, a modular switching fabric built from 4×4 non-blocking Collective Engines (CEs). Each CE combines spatial and temporal parallelism to perform reductions without accumulation buffers. CEs compose into k-ary n-tree topologies, scaling to 32- and 256-port switches while preserving non-blocking throughput. Source routing and flit-level synchronization keep per-switch state minimal. Our FPGA implementation shows that CCSwitch's quaternary-tree reduction fabric uses up to 23% fewer LUTs and 12–30% fewer flip-flops than a comparable Clos-based design at equal throughput. Enabling the full feature set—source routing, replication, and time-multiplexed VCs—uses 1.4–1.8× more LUTs than the circuit-switched baseline, well below the 3–5× overhead typical of packet-switched NoC routers, while supporting concurrent collectives on shared links.
Sumukh Pinge, Hardik Soni, Bob Lantz et al.· Proceedings of the ACM SIGCO...· 0 citations
Disaggregating the generation and training stages in RL is widely adopted to scale LLM post-training. There are two critical challenges here. First, the generation stage often becomes a bottleneck due to dynamic workload shifts and severe execution imbalances. Second, the decoupled stages result in diverse and dynamic network traffic patterns that strain the conventional static fabric. We build Silverstone to orchestrate dynamically both compute and network in disaggregated RL. Silverstone employs an adaptive compute scheduler that adjusts parallelism configuration to match changing workload characteristics within and across generation steps. Silverstone adopts a reconfigurable optical-electrical fabric called RFabric: It leverages optical circuit switches to reconfigure the aggregation and core layers of the topology on demand, tailoring bandwidth resources to the unique communication patterns across various phases of training, generation, and weight synchronization. Evaluated on a 64-H800 GPU testbed, Silverstone demonstrates up to a 1.42× throughput improvement over static baselines. Using a high-fidelity simulator, we also show that RFabric achieves superior performance-cost efficiency at scale over static Fat-Tree networks.
Xin Tan, Yicheng Feng, Yu Zhou et al.· Proceedings of the ACM SIGCO...· 0 citations
The Message Passing Interface (MPI) is a de-facto standard programming model for developing high-performance applications on large-scale supercomputers, where the performance of its collective communication operations over the underlying interconnection network is critical to the overall scalability of the applications. While the MPI defines the semantics of these operations, it leaves the algorithmic implementation to Open MPI, and selecting the algorithm typically relies on hard-coded decision logic. However, in real-world scenarios, this static approach often leads to sub-optimal performance, as it ignores the specific properties of the underlying system, particularly the complex and dynamic interconnection topologies. In this work, we address the challenge of selecting an efficient algorithm for a given collective, taking into account interconnection topologies. First, we demonstrate through a simulation study that interconnect topological features significantly influence the choice of the optimal collective algorithm for specific communication instances. Second, we propose a novel, machine learning-based, topology-aware framework for MPI collective algorithm selection. Third, we integrate this predictive framework directly into Open MPI, and show that our method is effective in selecting efficient collective algorithms for the Next Generation Tianhe Supercomputer, thereby unlocking the performance potential of the underlying interconnect.
Meng-Cheng Jia, Wei Zhang, Hui-Ze Zhang et al.· Fall Joint Computer Conferen...· 0 citations
Reconfigurable optical circuit switches revolutionize datacenter networks by allowing to adjust the physical topology in a dynamic and demand-aware manner. These switches directly match currently frequently communicating racks, reducing bandwidth tax and hence improving throughput. The underlying optimization problem is essentially the NP-hard Weighted k-Disjoint Matchings problem. Existing efficient solutions to this problem require a centralized controller, which constitutes a scalability bottleneck. This paper introduces the first distributed approach. Our main contributions are four new algorithms and a new approach to trade solution quality for running time via a parameter ε ∈ [0, 1]. Our best algorithm guarantees a \(\frac{1}{3}\)-approximation. We provide an extensive empirical evaluation on 87 real-world and synthetic workloads with billions of edges showing scalability and a speedup over state-of-art algorithms up to 1-2 orders of magnitude on most instances while retaining high-quality solutions.
Kathrin Hanauer, S. Heck, Stefan Schmid· International Conference on...· 0 citations
This paper presents a tutorial-style, collective-centric taxonomy for collective communication, which organizes recent advances into three layers: communication planning, which generates topology-aware collective schedules; communication execution and adaptation, which maps these schedules onto GPU runtimes and hardware in real clusters; and computation-communication coordination, which turns collective optimization into end-to-end training and inference benefits.
Xuebin Song, Menghao Zhang, Yue Liu et al.· IEEE Network· 0 citations