Title: Networking for Meta’s Gigawatt-scale AI Fleet
Authors: James Hongyi Zeng (Meta Compute / Infrastructure)
Introduction
This talk studies the networking challenges that arise when AI infrastructure grows from single-building GPU clusters to gigawatt-scale, cross-region AI fleets. The problem is important because modern AI systems require tens of thousands to hundreds of thousands of GPUs to communicate efficiently, while also dealing with power limits, geographic distribution, and continuous hardware failures. Existing datacenter networks fall short because they were not designed to simultaneously provide ultra-low-latency GPU-to-GPU communication, large-scale training fabric performance, and long-distance connectivity across campuses, regions, and public cloud facilities.
Key idea and contribution
The key idea is to divide Meta’s AI networking problem into three domains: scale-up, scale-out, and scale-across. Scale-up focuses on rack-level GPU interconnection, where dozens or hundreds of GPUs need to behave like one tightly coupled compute engine. This requires sub-microsecond latency, direct GPU memory access, very high bandwidth, and co-design across networking, optics, packaging, and accelerator hardware.
Scale-out focuses on connecting GPUs across racks and buildings, while scale-across focuses on connecting AI capacity across campuses, regions, and even national-scale WANs. Meta discusses both scheduled fabrics, which provide more predictable performance, and non-scheduled fabrics, which rely on standard switches and host-side optimization. A major contribution of the talk is its argument that open Ethernet-based approaches can become a practical foundation for large AI clusters, instead of relying only on proprietary interconnects.
Evaluation
The talk does not present a conventional experimental evaluation, but it provides evidence from Meta’s real deployment trajectory: 24K-GPU clusters in 2022-2023, 129K-GPU clusters across multiple buildings in 2024, and future cross-region AI supercomputers that may exceed 1GW of power. The evaluation is therefore based on operational scale, system design experience, and the practical constraints of deploying AI infrastructure at fleet level. This result is significant because it shows that AI networking must be evaluated by end-to-end training time, scalability, fault tolerance, and operational flexibility, rather than by isolated metrics such as link bandwidth or switch latency alone.
Personal thoughts
My main takeaway is that this talk clearly explains why networking has become a central bottleneck for large-scale AI systems. I like the scale-up, scale-out, and scale-across framework because it separates three different types of challenges while still showing that they must be solved together. It also highlights that AI infrastructure is not only a compute problem; it is also a networking, power, optics, and operations problem.
One open question worth exploring is how open Ethernet-based designs can maintain predictable performance under failures, heterogeneous accelerators, and cross-region deployments. I also think the operational side is especially important: at Meta’s scale, failures happen every day, so the network must support automatic rerouting, lossless upgrades, and hardware replacement without interrupting long-running training jobs.