Title: From Nimitz to NetPila: The Evolution of Production-Scale Container Network
Authors: Jiaqi Gao, Chao Qin, Sheng Cheng, Jiamin Cao, Guodong Yang, Zhenyu Zhang, Shuhong Zhu, Ennan Zhai, Dennis Cai (Alibaba Group)
Review：Ying Yao(SNG first-year PhD student)
Introduction
Containers have become a crucial deployment method for cloud services, search engines, big data, and AI training and inference. Container networking must provide connectivity, access control, and load balancing across a vast number of nodes while meeting three key requirements: cluster scaling must complete within minutes; network performance should approach that of physical networks; and both capabilities must remain stable at scale.
Alibaba launched its first-generation container network, Nimitz, in 2019. It utilized the industry-standard VxLAN protocol to decouple the container overlay address space from the physical network’s underlay address space. Cross-node packets required the addition of outer MAC, IP, UDP, and VNI headers; Open vSwitch (OVS) on each node maintained the overlay-to-underlay mapping and handled encapsulation, decapsulation, NAT, and forwarding.
While Nimitz supported dozens of services and thousands of tenants, the demands of AI workloads pushed single-tenant scales to the level of 100,000 containers and 10,000 nodes. At this scale, the design—in which every node stored a complete global mapping and every change was broadcast across the entire cluster—began to fail. The authors collectively refer to the elasticity, stability, and performance issues that arise when exceeding approximately 100,000 containers and 10,000 nodes as the “C100K problem.”
To address this, the authors began developing a second-generation system, NetPila, in 2022. Instead of optimizing VxLAN, NetPila eliminates the overlay-underlay mapping entirely. It encodes information regarding physical nodes, tenants, and containers directly into 128-bit IPv6 addresses, effectively making the container network a natural extension of the physical network and allowing it to directly leverage underlying IPv6 routing capabilities.
Key idea and contribution:
The paper first reviews how Nimitz deployed its VxLAN container networking into production, then illustrates its scalability limits through three real-world incidents, and finally introduces NetPila, which redesigns the data plane using a simpler IPv6 encoding scheme. The core of this evolution lies not in adding complex caching or hardware, but in eliminating the root causes of global state generation.
(1) Nimitz’s Five-Stage Data Plane
Directly mapping every pair of overlay and underlay addresses causes flow table size to balloon rapidly based on combinations of containers, nodes, and services. Nimitz decomposes the processing into five reusable stages: VxLAN decapsulation, VNI register assignment, service NAT, forwarding, and VxLAN encapsulation. This reduces flow table complexity from combinatorial growth to the sum of the counts of containers, nodes, and services, ensuring that changes to nodes or containers require modifications only at the corresponding stage.
Nimitz further employs Node CIDRs to aggregate containers on the same node into a small number of prefixes, allowing OVS to store only the mapping from CIDR to node IP. In a network comprising over 2 million containers and 25,000 nodes, this optimization shrinks the pipeline scale by approximately 80-fold. Active “MegaFlows” are dynamically offloaded to the NIC; once a hardware hit occurs, subsequent packets bypass OVS, enabling a single spare CPU core to sustain 400 Gbps line-rate traffic.
However, the dynamic offload channel supports only about 638 entries/s, which may fall short of the ~1,000 entries/s demand during traffic spikes like major sales events. The team relies on pre-warming via traffic simulation to mitigate known peaks, but this does not eliminate the scalability issues inherent in the global mapping itself.
(2) Three Symptoms of the C100K Scale
The first symptom is a slowdown in scaling operations. When expanding a 25,000-node cluster by 10%, new nodes must fetch the full mapping while existing nodes must retrieve information about the new nodes, resulting in approximately 150,000 Orchestrator “list” queries. Given a limit of 200 QPS per tenant, the theoretical processing time is at least 12 minutes, while the actual warm-up time observed by tenants can reach 20 minutes. Cleanup is similarly difficult: when OVS holds 50,000 rules, the time required to delete a batch of 2,000 rules increases by approximately 28-fold compared to a smaller table.
The second symptom is RDMA performance fluctuation. A new training job spanning 12,000 nodes required all nodes to install a global mapping, triggering OVS revalidation. Because hardware counter reporting can be delayed by up to two seconds, newly installed RoCE table entries were mistakenly identified as idle and deleted, only to be re-installed immediately, creating a repetitive offload-remove cycle.
The third symptom is unavoidable virtualization overhead. Even though encapsulation and decapsulation are handled by the NIC, the VxLAN header causes a ~5% drop in RoCE throughput for large messages; hardware encap/decap increases latency for small messages by ~5%, ultimately resulting in a ~3% decrease in multi-GPU inference throughput.
The authors experimented with caching encapsulation results, adding programmable hardware, mapping sharding, lazy loading, and delayed cleanup, but none of these approaches could eliminate the dependency on global mapping. Cache misses would fall on the critical communication path, and initial packet failures or delays during RoCE connection establishment are unacceptable in a production environment.
(3) Unifying container and physical networks using IPv6
Further research by the team revealed that large-scale AI and cloud-native workloads rarely utilize underlying VxLAN capabilities such as custom subnets, BYOIP, or IP migration. Training frameworks typically rely on a master node to collect and broadcast participant addresses, while service meshes expose endpoints via load balancers or DNS. What tenants truly require is connectivity, ACLs, and load balancing; thus, address flexibility can be partially sacrificed to achieve greater scalability.
NetPila structures container addresses as 128-bit IPv6 addresses: the first 64 bits represent the node address (routable within the physical network), the middle 24 bits represent the tenant ID, and the final 40 bits represent the container address (unique within the node). The physical network needs only to examine the first 64 bits to route packets to the target node, while the node-level agent maintains only local container state, eliminating the need to broadcast changes across the entire cluster.
NetPila’s data plane has also been simplified from Nimitz’s five-stage process to just two stages: service NAT and forwarding. By eliminating the need for VxLAN, gateways, edge switches, and global overlay-underlay mappings, the flow table scale is reduced from a state dependent on the number of nodes to one dependent on the number of local containers. All rules can be statically offloaded to the NIC, removing the agent from the per-packet processing path.
(4) Isolation and Address Privacy
IPv6 encoding embeds node location and tenant ID information directly into the address. Consequently, NetPila assigns a unique key to each tenant and uses AES-128 to transform plaintext IPs into ciphertext IPs visible to the containers. For standard communication, the initial packet undergoes decryption and mapping in software; the resulting translation entry is then installed into the hardware via tc, ensuring subsequent packets avoid software-path latency. The data plane also validates the tenant IDs in both source and destination addresses, dropping any traffic that lacks explicit authorization by default.
For RoCE, NetPila avoids per-packet IP rewriting in favor of GID replacement within the NIC driver: containers see the ciphertext GID while the host retains the plaintext GID. During connection establishment, the driver swaps these values and reuses the QP hardware context. This approach introduces no initial-packet latency and consumes no additional hardware table entries.
Evaluation
The authors compared Nimitz, NetPila, and non-virtualized Native IPv6 in a production GPU cluster. Each GPU server was equipped with 8 GPUs and 4 NICs, with GPUDirect RDMA enabled. The evaluation covered four aspects: rule scale, cluster scalability, RoCE performance, and end-to-end LLM inference.
(1) Rule Count and Scalability
Two production workloads spanned 8,076 nodes and 6,176 nodes, respectively; Nimitz’s peak hardware offload entries approached 8,000 per node. In contrast, NetPila required only 12–20 rules per node at the same scale, all of which could be statically offloaded to the hardware.
With a fixed deployment of 31 containers per node, the number of entries per node in Nimitz grew linearly with cluster size, while the messages and time required for a 10% scale-out grew quadratically. For NetPila, the number of entries per node remained stable, and scale-out time increased approximately linearly. At a scale of 25,000 nodes, a 10% scale-out took approximately 12.5 minutes for Nimitz but only 1.25 minutes for NetPila; at 100,000 nodes, NetPila required about 5 minutes for a 10% scale-out.
(2) RoCE Latency and Bandwidth
Perftest results showed that, compared to Nimitz, NetPila achieved a maximum latency reduction of 4.281% and a maximum bandwidth increase of 6.055% for RDMA Read, Write, and Send operations. The performance difference between NetPila and Native IPv6 fell within the margin of measurement error, indicating that removing VxLAN allowed the container RoCE network to achieve performance essentially matching that of the physical IPv6 network.
(3) LLM Inference
The authors deployed Llama-3.2 across two servers and conducted tests using vLLM. Compared to Nimitz, NetPila improved throughput for text generation requests, output tokens, and total tokens by approximately 0.854%, 0.863%, and 0.862%, respectively; meanwhile, average TTFT, TPOT, and ITL decreased by approximately 0.800%, 0.669%, and 1.081%, respectively. Although the magnitude of improvement is modest, it demonstrates that network virtualization overheads impact end-to-end AI applications.
NetPila has been in production for over two years. A representative deployment comprises 9,205 nodes and 589,120 containers, with production evaluations covering tasks involving up to 25,000 nodes; the system is designed to support millions of containers for a single tenant.
Personal thoughts
The most noteworthy aspect of this paper is not the superficial shift from VxLAN to IPv6, but rather the insight gained from five years of production experience: abstractions initially introduced to provide flexibility can become a major cost burden as system scale and business requirements evolve. Although Nimitz was heavily optimized through techniques such as pipeline splitting, CIDR, hardware offloading, and pre-warming, the fundamental need for global mapping and cluster-wide broadcasting persisted because the overlay could not route traffic independently.
NetPila’s design reflects deliberate trade-offs. It sacrifices certain capabilities—such as custom addressing, IP migration, and overlay isolation—to achieve localized state, static hardware rules, and RoCE performance approaching that of bare-metal environments. This approach is well-suited for large-scale container workloads where membership is managed by training frameworks, service discovery, and load balancers, though it may not be appropriate for VPCs requiring rich address semantics.
Furthermore, NetPila’s simplicity relies on several prerequisites: the physical network must support IPv6, the platform must allow modifications to agents and NIC drivers, and the system must be capable of managing address encryption keys. Migrating legacy IPv4 applications still necessitates IPv4-to-IPv6 translation. Nevertheless, based on metrics such as rule count, scaling time, and over two years of production data, the paper compellingly demonstrates that in specific scenarios, “eliminating a layer of mapping” can be more effective than continuing to optimize the mapping itself.