Jul 2026· Fall Joint Computer Conference· pp. 169-176· 0 citations· 30 references
Abstract
Torus networks are deployed in production AI training clusters for their path diversity and low latency, but 2D Torus scales poorly: electrical packet switches compromise latency, and high-dimensional Torus introduces excessive routing complexity. We present STON (Scalable TOrus Network), a hierarchical architecture that treats a 2D Torus as a supernode and interconnects supernodes with a reconfigurable Optical Circuit Switch (OCS) for AlltoAll-dominated large-scale training networks. STON comprises three coordinated modules: (1) fragmentaware task placement, which minimizes inter-supernode traffic by reducing job fragmentation; (2) non-disruptive logical topology mapping, governed by two principles that prevent OCS reconfiguration from disrupting running tasks or partitioning multisupernode jobs; and (3) compute-phase traffic forwarding, which ensures reachability when direct OCS circuits are unavailable. STON reduces average FCT by 42.2%-61.1% across synthetic workloads and by 52.6% on a one-day Kalos production trace (under an AlltoAll traffic model for all jobs), with 95th-percentile tail latency reduced by up to 74.5%, versus a static direct-connect baseline using the same OCS hardware.
Collective communication operations in AI and HPC workloads generate heavy network traffic. Offloading these operations to network switches reduces latency, but performing arithmetic and replication at line rate is difficult, especially as port counts and link speeds grow. Existing in-network approaches rely on accumulation buffers that not only limit throughput but also require complex state management to handle stragglers and congestion. We present CCSwitch, a modular switching fabric built from 4×4 non-blocking Collective Engines (CEs). Each CE combines spatial and temporal parallelism to perform reductions without accumulation buffers. CEs compose into k-ary n-tree topologies, scaling to 32- and 256-port switches while preserving non-blocking throughput. Source routing and flit-level synchronization keep per-switch state minimal. Our FPGA implementation shows that CCSwitch's quaternary-tree reduction fabric uses up to 23% fewer LUTs and 12–30% fewer flip-flops than a comparable Clos-based design at equal throughput. Enabling the full feature set—source routing, replication, and time-multiplexed VCs—uses 1.4–1.8× more LUTs than the circuit-switched baseline, well below the 3–5× overhead typical of packet-switched NoC routers, while supporting concurrent collectives on shared links.
Sumukh Pinge, Hardik Soni, Bob Lantz et al.· Proceedings of the ACM SIGCO...· 0 citations
Dragonfly-Ultra is presented, a scalable, low-cost network architecture for high-performance AI clusters that can scale to over 260k GPUs with only 82% cost and 81% power consumption of a 3-layer Clos architecture and incorporates three key mechanisms to further improve network performance and optimize collective communication.
Rui Zhuang, Hui Yuan, Junye Zhang et al.· Proceedings of the ACM SIGCO...· 0 citations
ReTri, a bidirectional All-to-All schedule for ORNs based on the Trivance algorithm is presented, a bidirectional All-to-All schedule for ORNs based on the Trivance algorithm that improves completion time and improves reconfigurable Bruck by up to 2.1×.
Anton Juerss, Stefan Schmid· Proceedings of the ACM SIGCO...· 0 citations
Optical-circuit-switched interconnects have become one of the core components for AI training due to their flexible topology reconfiguration. In contrast to the applications carried by traditional data center networks, large-scale language model training is highly sensitive to network failures, where frequent disruptions will cause gradient synchronization delays, leading to training interruptions and wasted computational resources. Existing schemes are primarily focused on specific communication patterns, without considering the fault probability distribution. As a result, unreliable links remain on critical paths. Furthermore, passive fault response mechanisms lead to inefficient topology reconfigurations, preventing network protocol convergence and making it difficult to meet the stringent stability requirements of large-scale model training. To address reliability challenges in optical-circuit-switched interconnect, we propose TopoCrafter, which leverages dual-agent deep reinforcement learning to proactively mitigate network failures. The “Topo-Agent” estimates link failure probabilities to determine reconfiguration timing and then employs a lightweight heuristic algorithm to create failure-avoidant topology that matched to traffic pattern. Concurrently, the “Route-Agent” optimizes traffic distribution. Through their strategic interaction, the agents learn holistic policies that optimally balance network reliability and communication efficiency. To improve generalization, a progressive training approach is employed, allowing the agents to adapt to complex failure environments while accelerating convergence. Under link failure scenarios, TopoCrafter maintains reliability, reducing end-to-end latency by up to 50% and maximum link utilization by approximately 20% compared to FatTree. In addition, progressive training algorithm ensures a performance degradation of less than 10% when adapting to new failure environments, and it maintains stable high performance as the network scales.
Liang Qin, Xingyu Liu, Wenting Wei et al.· IEEE Transactions on Cogniti...· 0 citations
This work proposes a novel topology named the Balanced Sparse Tree (BST), which is a topology characterized by symmetric design and sparse connections, motivated by hypergraph theory and Steiner Systems, and demonstrates the superiority of BST over the state-of-the-art in network scale, latency, bandwidth, and cost.
Shaoteng Liu, Dejun Kong, Huitian Wang et al.· 0 citations
The proposed Multi-Path Multi-Level Feedback Queueing (MP-MLFQ) leverages the spatial diversity and regularity of DCNs to realize a scheduler with numerous logical priority levels while occupying as low as 2 physical priority queues within network switches.
Alessandro Cornacchia, Andrea Bianco, Paolo Giaccone et al.· 0 citations