Title: GGN: Experiences in Designing and Deploying the Next-Generation Google Global Network
Authors: Mohammad Al-Fares, Anees Shaikh, Arjun Muralidharan, Bikash Koley, Charith Wickramaarachchi, Chi-Yao Hong, Jonathan Zolla, Min Zhu, Namrata Kadam, Priya Mahadevan, Subhasree Mandal, Rob Shakir, Aman Shaikh, Arda Balkanay, Arjun Singh, Dennis Fetterly, Nachikethas A. Jagadeesan, Richard Alimi, Sankalp Singh, Warren Martins (Google)
Reviewer: Xing Fang (SNG fifth-year PhD student)
Introduction
Written by Google’s networking team, this paper presents the design and multi-year deployment experience of the Google Global Network (GGN), its next-generation global WAN. GGN is not a local upgrade to either B2 or B4. Instead, it gradually brings the two previously independent backbones into a unified software-defined network that is modular, sharded, regionalized, and hardware-agnostic. The goal is to give a global WAN built for cloud services and AI/ML training smaller failure domains, sustainable horizontal scaling, faster technological evolution, and explicitly enforceable availability SLOs.
For more than a decade, Google primarily operated two global WANs. B2 was a traditional carrier-grade IP/MPLS network serving Internet-facing traffic, while B4 was a centrally controlled SD-WAN carrying large volumes of inter-datacenter backend traffic. This dual-backbone design allowed each network to optimize for a different objective—high availability for B2 and high utilization for B4—but its limitations became increasingly apparent as Google’s workloads changed.
First, the two networks had separate control planes, management systems, and operational processes. Features such as IPv6 and QoS often had to be developed and deployed twice, increasing engineering cost and slowing network evolution. Second, both B2’s distributed MPLS-TE control plane and B4’s centralized TE system were approaching their scaling limits. B4’s global control plane was especially risky because a software defect or bad input could affect multiple regions at once. Finally, the two networks could not share capacity flexibly: during a submarine-cable failure or sudden traffic spike, one backbone could become congested while the other still had spare bandwidth.
Figure 1: The continued growth of MPLS-TE LSPs in B2 and tunnels and forwarding entries in B4 pushed both control-plane and hardware-table limits.
At the same time, Google Cloud customers and cross-datacenter AI/ML training introduced stricter requirements. Enterprise customers wanted predictable bandwidth, latency, and availability, as well as control over the geographic regions their traffic traversed. Large-model training drove network capacity growth even faster. The traditional approach of scaling vertically with larger devices and more powerful monolithic controllers could no longer keep pace, prompting Google to redesign its global WAN around a unified architecture.
Core Ideas and Contributions
The first challenge was preventing global failures. A global controller can optimize the entire network, but it also creates an enormous failure domain. The paper’s central insight is that high availability should not rely only on making software more reliable; the architecture itself must limit how far any one failure can propagate. GGN therefore combines core sharding with control-plane regionalization, turning potential global outages into events contained within one shard or one region.
The second challenge was capacity scaling. With cloud and AI/ML traffic growing rapidly, neither faster ports nor a larger single network domain could scale indefinitely. GGN replicates the full global core into several independent shards, each with its own data and control planes. Adding a shard increases both capacity and redundancy, shifting the network from vertical scaling of a monolith to sustainable horizontal scaling.
The third challenge was enabling modules to evolve independently. In the old architecture, edge networks, core networks, and device implementations were tightly coupled. GGN divides the network into functional domains—including the Global Core, Regional Core, Compute Edge, Customer Service Edge, and Internet Edge—and exchanges only the necessary routing and bandwidth information through the Edge-Core API. As long as the interface remains stable, each domain can evolve independently while containing errors within its own boundary.
Figure 2: GGN uses functional domains and the Edge-Core API to provide modularity and fault containment, with edge domains attached to a sharded core.
The fourth challenge was keeping production traffic safe during a migration lasting several years. Google could not replace the globally deployed B2/B4 backbones through a single cutover. The paper therefore treats incremental deployment and rollback as architectural properties rather than temporary operational measures: the old and new networks must coexist for an extended period, and traffic must be movable gradually by application, destination, or even individual flow.
Under this design, the core domains forward traffic among different edge domains, with the Global Core connecting multiple Regional Cores. Compute, customer-service, and Internet edges provide different types of ingress and egress. Domains exchange a compact set of reachability information through an eBGP-based routing interface, while a BwE-based bandwidth interface communicates capacity allocations to the edges. These narrow interfaces reduce coupling and prevent abnormal state from one edge domain from spreading into the core or other edge domains.
Core sharding. GGN treats B4 as its first shard and adds three parallel Global Core shards. The four shards are physically isolated at Layer 2 and above, and each has independent routing, traffic-engineering, and BwE control systems. Edge domains connect to every shard, and edge routers distribute traffic among them. Capacity is planned for N-1 failures: even if an entire shard fails, the remaining shards should absorb protected high-priority traffic without violating its SLO.
Figure 3: Core-sharding architecture. Traffic is split across independent shards at the source edge and merged again at the destination edge.
Cross-shard control is handled by the Shard Enforcer (SE). It aggregates reachability, available bandwidth, and service SLOs from each shard, then distributes shard-selection decisions to edge routers through iBGP. Recovery occurs at three timescales: the data plane and PRR avoid blackholed paths within sub-seconds, BGP withdraws an unreachable shard within seconds, and SDN controllers recompute global routes and bandwidth over minutes. Tier-0 and Tier-1 traffic is moved to healthy shards first, while interruption-tolerant, lower-priority jobs may be temporarily rate-limited rather than consuming capacity reserved for critical workloads.
Regionalization. Sharding primarily isolates cross-shard and global failures; regionalization further reduces the geographic failure domain. Each region runs independent TE and BwE controllers that read and control only local topology and traffic. Intra-region traffic is fully managed by regional controllers, while inter-region traffic is jointly handled by the Regional and Global Cores. The two controller types share some physical resources at boundary nodes, but link capacity is logically partitioned. If their intents conflict, the boundary node gives priority to the regional controller, preventing a global-control error from disrupting local traffic.
Figure 4: Regionalized architecture. Regional and Global Cores share boundary nodes while keeping controllers, input data, and logical bandwidth isolated.
Hardware optionality. GGN also uses the WAN Building Block (WBB) to avoid dependence on one custom platform or vendor implementation. Google defines an abstract hardware specification, and any vendor that meets the functional, throughput, and power requirements can supply interchangeable devices. Management and control use OpenConfig interfaces such as gNMI, gNOI, and gRIBI. Different shards use different vendors or software versions, reducing the chance that a shared vendor defect, supply-chain problem, or faulty upgrade takes down every shard simultaneously.
Incremental migration. GGN migrates traffic using host-side markings. A host can direct traffic to the legacy network or spread it across new shards by application, destination, or flow. Before each migration, automation checks the target shards’ capacity, L3/L7 availability, and latency SLOs, and reserves bandwidth in advance. Migration proceeds week by week for individual source–destination cluster pairs, while preserving the ability to roll back quickly to B2. New shards are filled primarily by absorbing organic traffic growth, reducing the cost of building duplicate capacity.
Experimental Evaluation
GGN’s evaluation is based on years of production deployment rather than a single offline experiment. By July 2026, the new shards 2–4 accounted for 35% of total GGN capacity, and up to 40% of traffic in one of Google’s largest AI/ML regions was already using sharded transport. Relative to B4 in 2019, today’s GGN has roughly 45 times the total capacity, while each new shard is itself about five times the size of the 2019 B4 network. On average, Google has been adding the equivalent of a 2019-scale B4 every two months, with the most recent pace approaching one every two weeks.
As Active Path Diversity, PRR, and automated TE-to-Routing Fallback were introduced, every GGN region pair consistently achieved a user-traffic availability SLO above 99.99% over the past two years. Regionalization currently covers 53% of network regions and protects 12% of intra-region traffic. Since January 2024, total intra-region traffic has grown by 406%, while traffic protected by regionalization has grown by 947%.
Production incidents further demonstrate the value of containment. During one shard-routing failure, BGP and PRR quickly moved user traffic away from the faulty paths, after which SE disabled the shard; users saw only a brief, minor loss event. In another incident, a capacity problem overloaded the TE controller of one shard. SE switched shards only for high-priority traffic, allowing lower-priority traffic to consume its availability budget through rate limiting. Regionalization similarly protected completed regions during two incidents involving a faulty path policy and corrupted topology input: non-regionalized areas suffered visible packet loss, while regional controllers remained unaffected by the global errors.
Figure 5: Recovery and containment across four production incidents. Sharding reduces the impact of single-shard failures, while regionalization protects selected regions from global configuration and input errors.
The paper also quantifies the architectural cost. Compared with continuing to develop a hypothetical next generation of custom hardware, WBB is approximately cost-neutral over five years and produces about 2% net-present-value savings. Physical core sharding increases overall cost by roughly 2%–5% when failure protection is reserved for critical traffic, while regionalization adds about 1% in network cost. Fault containment is therefore not free, but its cost is far below the potential impact of a global network outage.
Q&A
Q1: How does GGN handle a physical failure that could affect multiple shards at once, such as a regional power outage?
A1: The shards are designed to be physically disjoint. Each shard is also deployed across multiple physical failure domains, including separate power infrastructure. The likelihood of all shards simultaneously experiencing multiple correlated physical failures is therefore extremely low.
Q2: Do different shards run on separate physical devices and maintain their own control and data planes?
A2: Yes. Every shard has an independent control plane and data plane implemented on separate physical devices. Different shards also use equipment from different vendors, so they are not homogeneous implementations.
Q3: Do these independent control planes compare or validate the routing or forwarding states they eventually compute?
A3: No. The control planes operate independently and do not communicate with one another, nor are they required to produce identical routing or forwarding states.
Personal thoughts
During the Q&A, an attendee asked whether a regional power outage or similar physical event could affect multiple shards at once. The speaker emphasized that shards are physically disjoint and that each shard spans multiple failure domains, including power infrastructure, making a correlated failure across all shards highly unlikely. This answer also highlights that GGN’s reliability comes not merely from “having four shards,” but from systematic isolation across physical failure domains, control planes, and vendor implementations. Those isolation boundaries must continue to be maintained as the network expands and evolves.
GGN is also built on Google’s exceptional network scale, software capabilities, and long-term infrastructure investment. Building several physically isolated global networks—together with their control, testing, and migration systems—may not be economical for smaller operators. The most transferable lesson may not be to reproduce all four shards, but to choose isolation boundaries appropriate to one’s own scale and determine whether the reliability gain justifies the additional construction and operational cost.