Skip to content
Book Open access

From Nimitz to NetPila: The Evolution of Production-Scale Container Network

Aug 2026 · Proceedings of the ACM SIGCOMM 2026 Conference · pp. 883-896 · 0 citations · 25 references

TL;DR

By embedding container addresses in the 128-bit IPv6 space, NetPila integrates the container network with the physical network, reducing packet-processing complexity and improving scalability.

Abstract

This paper describes two generations of production container networks deployed for over five years. Our first-generation network, Nimitz, used VxLAN for overlay-underlay mapping, providing flexibility and scalability from 2019-2022. However, with the rise of large-scale services (e.g., AI training and inference), Nimitz hit the C100K problem: beyond O(100K) containers, address mapping overhead and complex packet processing caused significant performance degradation. To overcome this, we built NetPila, a second-generation design that uses simple, practical IPv6 addressing to remove VxLAN encapsulation and overlay-underlay mapping tables. By embedding container addresses in the 128-bit IPv6 space, NetPila integrates the container network with the physical network, reducing packet-processing complexity and improving scalability. NetPila now supports millions of containers per tenant for production AI training and inference. We present the design, lessons, and deployment results of both generations.

Read PDF

Similar papers

Book Open access Aug 2026

GGN: Experiences in Designing and Deploying the Next-Generation Google Global Network

Google's Global Network (GGN), a major architectural redesign of the WAN that evolves B2 and B4 into a single, modular, and highly available software-defined network, is presented.

Mohammad Al-Fares, R. Alimi, Arda Balkanay et al. · 0 citations
2026

Rethinking Virtual Network Construction for Network Emulation at Scale: Analysis, Modeling, and Optimization

Network emulation has become an indispensable methodology for evaluating next-generation network architectures, offering a critical balance between experimental fidelity and operational scalability. However, its effectiveness is fundamentally constrained by inefficiencies in emulating large-scale networks, particularly during virtual network construction. This bottleneck arises from mandatory serialization of virtual link instantiation and operating system (OS) kernel-level notification overheads, which collectively degrade performance by orders of magnitude on 10K-node topologies. Departure from the current practice that employs a multi-machine framework for improvements, we propose SplitNN (Split Network and Namespace), a novel single-machine network emulation paradigm that breaks the serialization constraint through multi-VM (virtual machines) partitioning, and reduces notification overheads via namespace segmentation. Extensive evaluations show that SplitNN constructs 10K-node virtual networks within 1–5 minutes on a single machine, achieving a 98.5%–99.2% reduction in construction time compared to state-of-the-art emulators. While primarily a single-machine solution, SplitNN seamlessly integrates with multi-machine deployments, complementing them by enabling cumulative gains in both scalability and efficiency.

Kaifei Peng, Yanbiao Li, Wenbin Li et al. · 0 citations
Book Open access Aug 2026

Pegasus: A Data Center Network for Bare-Metal AI Cloud

The experience in designing, deploying, and operating Pegasus, a data center network tailored for the AI cloud, along with operational lessons learned from its deployment are shared.

Xianneng Zou, Yadong Liu, Yiran Zhang et al. · 0 citations
Open access Jul 2026

Virtual Firewalls Scaling and Placement with Traffic Distribution in Telco Cloud-Edge Continuum

The evolution of the 5 G and future 6 G networks into a virtualized infrastructure enables the deployment of virtual firewalls (vFW) to protect the network from undesirable traffic and other threats. The main advantages of the vFW systems come from the flexible deployment of vFW instances across the Telco Cloud-Edge Continuum (TCE) infrastructure and from horizontal scaling of vFW to accommodate daily changes in traffic demand. We propose a new virtual Firewall Allocation and Traffic Distribution (vFATD) approach to orchestrate the vFW system. We formulate the vFATD problem, design a MILP-based optimum orchestration algorithm, and propose k-center-based and genetic-evolution-based heuristic algorithms for practical use. Comprehensive experiments based on actual network topologies and traffic data from a mobile network operator confirmed that the proposed vFATD approach significantly outperforms the current approaches. The results say that the main gain comes from engaging traffic distribution that: i) reduces the vFW system costs by optimizing used computing and network resources, ii) improves its robustness against unexpected traffic changes, e.g., during DDoS attacks, and iii) relaxes the need for precise vFW provisioning as its performance is continuously adapted to actual traffic demands.

Bartosz Kopeć-Persiński, Andrzej Bęben · 0 citations