Jul 2026· International Conference on Supercomputing· pp. 314-327· 0 citations· 54 references
Computer Science
TL;DR
This work introduces Calculon-MoE, a tool that jointly explores FLOPS, HBM bandwidth and capacity, multiple network topologies, and popular parallelism/optimization strategies used in LLMs, and quantifies the benefits of overlapping compute and communication, hardware-accelerated collectives, widening the scale-up domain, and higher memory bandwidth and capacity.
Abstract
The massive compute, memory, and networking needs for LLM training necessitate a fundamental rethinking of datacenter architectures to ensure scalability, efficiency, and cost-effectiveness. In particular, the design of the network fabric for AI datacenters for emerging LLMs (such as MoEs) remains a crucial and challenging open question, spanning technology choices (that determine the size of the high-bandwidth domain), topology, and software optimizations (collective algorithms and overlap strategies). This necessitates an agile framework to traverse the co-design space. This work introduces Calculon-MoE, a tool that jointly explores FLOPS, HBM bandwidth and capacity, multiple network topologies (Two-tiered vs. FullFlat optical), the size of scale-up domain, and popular parallelism/optimization strategies used in LLMs. Our validation studies demonstrate that our LLM/MoE runtime predictions are within 10% of real-world measurements. Using Calculon-MoE, we conduct a suite of case studies to develop an actionable roadmap for data centers. For example, the results point to the promise of Fullflat network architectures, which provide uniform high-bandwidth, low-latency connectivity between all nodes and demonstrate their positive impacts on performance and scalability. We also quantify the benefits of overlapping compute and communication, hardware-accelerated collectives, widening the scale-up domain, and higher memory bandwidth and capacity. Our study spans both sparse (mixture of experts) and dense transformer-based LLMs, revealing how system design and optimization choices affect system efficiency and overall throughput in both cases.
Google's Global Network (GGN), a major architectural redesign of the WAN that evolves B2 and B4 into a single, modular, and highly available software-defined network, is presented.
Mohammad Al-Fares, R. Alimi, Arda Balkanay et al.· Proceedings of the ACM SIGCO...· 0 citations
The arrival of 100K+ GPU clusters marks a new frontier in AI infrastructure. Standard communication stack meets new challenges as physical topologies span multiple datacenter buildings, introducing high bandwidth-delay product links where latency increases by up to 30× compared to intra-rack traffic. Furthermore, the transition toward Mixture-of-Experts architectures generating bursty all-to-all patterns that create transient congestion hotspots. These constraints, combined with an operational environment where hardware failures shift from anomalies to frequent occurrences, renders traditionally lightweight operations like initialization and resource management challenging. We present Meta's network architecture and software stack designed to support one of the world's largest RoCE fabrics, currently connecting over 100,000 GPUs across multiple datacenter buildings. To overcome scaling barriers, we introduce a scalable initialization strategy that reduces startup times by 11× via eager process group creation and O(N) topology discovery, alongside a resource management system that cuts GPU memory usage by 2× through on-demand allocation. We further detail a custom transport layer utilizing Dynamic Queue Pair Load Balancing to saturate links, and a set of operation toolings. These innovations have been deployed in production, providing the foundational communication fabric for training state-of-the-art Large Language Models.
Hongyi Zeng, Min Si, Pavan Balaji et al.· Proceedings of the ACM SIGCO...· 0 citations
The deployment of Mixture-of-Experts (MoE) models on production high-bandwidth superpods, such as NVIDIA's NVL72/576 and Huawei's CloudMatrix384, introduces critical challenges beyond raw interconnect bandwidth. While these systems provide unified global address spaces and high-bandwidth fabrics, their full potential for sparse MoE communication is hindered by three fundamental bottlenecks: (1) Strict execution serialization imposed by coarse-grained Bulk Synchronous Parallel (BSP) orchestration of interdependent communication phases; (2) Prohibitive synchronization overhead that fails to scale alongside high interconnect bandwidth; and (3) Severe load imbalance resulting from distance-agnostic scheduling of irregular token traffic. To eliminate these bottlenecks, we introduce UBEP (Unified-Bus Expert Parallelism), a production-ready communication library that rethinks MoE's All-to-All primitives for modern superpod architectures. Through large-scale experiments, UBEP reduces All-to-All latency by up to 52.4% and MoE inference Time Per Output Token (TPOT) by up to 11.1%.
Yipeng Liu, Chang Liu, Sitan Shen et al.· Proceedings of the ACM SIGCO...· 0 citations
Networking research advances by turning hypotheses into empirical evidence, so accelerating it means reducing the lag between ideation (synthesizing a hypothesis) and generating the data that tests it. Consider a concrete case: does a bulk BBR download fairly share its bottleneck with competing real-time Google Meet traffic? Validating this requires configuring a realistic bottleneck link, concurrently generating BBR's bulk transfer and Meet's real-time traffic, and collecting relevant service-quality metrics. Today this overhead is high, often forcing researchers to start from scratch for every new idea. This ideation-to-data-generation gap will only worsen in the agentic AI era, where AI-assisted ideation accelerates exponentially, yet its outputs cannot be validated without a data-generation backend. This paper explores how to bridge this gap. We envision a composable, domain-specific backend, Pramana, shaped as a thin waist, with diverse research intents at the top and disparate execution substrates at the bottom. Pramana realizes this waist through a single contract, the intent specification, which disaggregates an experiment into three independent axes: the intent (what data to generate), the substrate (where to generate it), and the mechanism (how to produce it), so one specification runs on any substrate. We demonstrate Pramana's utility by building a first-of-its-kind corpus of 255 data-generation intents mined from 66 published papers, and show the intent specification satisfies all of them, where no existing tool satisfies more than 13%. Our current proof-of-concept implementation already satisfies 34% of these intents, more than twice the best existing tool, and we lay out a roadmap for closing this abstraction-implementation gap through a broader community effort to build the envisioned data-generation backend and accelerate empirical networking research.
Jaber Daneshamooz, Eugene Vuong, A. Ramanathan et al.· 0 citations
Aim: This study aimed to extend the staged-rollout risk framework proposed by Zhao et al. (2018) to multi-datacenter environments by accounting for failure-domain topology, per-datacenter sequential monitoring, and automated rollback. The resulting framework, termed the Isolation Deployment Protocol (IDP), was designed to address the limitations of using traffic-percentage exposure as a proxy for failure exposure in distributed systems with shared control-plane services, replicated caches, and cross-region service-discovery dependencies.
Methods: The Isolation Deployment Protocol (IDP) was developed by integrating four components: a real-time Failure Domain Graph Model, a Topology-Aware Ramp-Up (TARU) algorithm, per-datacenter Sequential Probability Ratio Test (SPRT) monitoring, and automated topology-aware rollback. IDP was evaluated on a six-node bare-metal Kubernetes testbed running Istio and Consul, with namespace isolation used to represent separate datacenters. Twenty independent trials were conducted against a topology-blind baseline, measuring realized blast radius and Mean Time to Containment (MTTC). The blast-radius model was also validated through formal analytical analysis.
Results: Formal analysis demonstrated that the topology-blind baseline could underestimate blast radius by up to 8.62× in the evaluated failure scenario. Across 20 independent trials, IDP reduced the mean realized blast radius by a factor of 13.0 compared with the topology-blind baseline, restricting error propagation to the targeted 5% canary namespace. IDP also reduced Mean Time to Containment by 94.4%, from 142.4 minutes under global metric aggregation to 7.9 minutes with per-datacenter monitoring. These findings demonstrate that deployment decisions based on structural failure domains can substantially limit the propagation of failures beyond the intended canary population.
Conclusion: These results indicate that topology-aware rollout control and per-datacenter monitoring can substantially reduce failure propagation and containment time relative to topology-blind global monitoring.
Recommendation: The experiments ran within a single bare-metal host, and production multi-region network latency was not modeled; this represents an open limitation of the current evaluation. Future research should validate IDP across geographically distributed datacenters under realistic inter-region latency, packet loss, traffic variability, and production-scale dependency graphs.
Sai Joshitha Kathari· American Journal of Technolo...· 0 citations
AI training and inference are driving cloud networks toward terabit-per-second (Tbps) bandwidth per server, challenging the scalability and efficiency of today's cloud network architectures. A prevalent design scales bandwidth by stacking monolithic Data Processing Units (DPUs), but this approach tightly couples control and data plane resources, leading to excessive cost, power consumption, and operational complexity. We identify a fundamental control-data plane divergence in AI clouds: while data plane bandwidth demand grows rapidly, control plane demand remains largely flat due to the dominance of elephant flows. As a result, monolithic DPUs become systematically over-provisioned when used as bandwidth scaling primitives. We present DistDPU, a disaggregated DPU architecture that redefines the scaling abstraction for high-bandwidth cloud networking. DistDPU decomposes a monolithic DPU into lightweight, bandwidth-provisioning Execution Modules (EMs) and a shared, control-centric Orchestration Module (OM), enabling independent scaling of data and control plane resources. By scaling out low-cost EMs under a single OM, DistDPU exposes a unified, high-bandwidth logical DPU interface to the cloud management plane. To preserve RDMA performance and multi-tenant isolation at scale, we co-design the EM-OM functions and the inter-module fabric to minimize virtualization overhead while enforcing security and manageability invariants equivalent to those of a monolithic DPU. DistDPU has been deployed in production for two years. It serves more than 10,000 GPUs and delivers higher efficiency and strong performance on real-world AI workloads than state-of-the-art designs.
Hao Mei, Lizhou Gao, Yuanyi Zhu et al.· 0 citations