Title: Pegasus: A Data Center Network for Bare-Metal AI Cloud
Authors: Xianneng Zou (Tsinghua University); Yadong Liu (Tencent); Yiran Zhang (Unaffiliated); Jian Zhao, Zhaoxun Zhou (Tencent); Qing Wang (Nanjing University); Xingyi Li, Liang Wang (Tencent); Xingda Wei (Shanghai Jiao Tong University); Xiaojie Huang, Zhaohe Chen, Yinben Xia (Tencent); Yuxi Wang (Unaffiliated); Kai Ren, Lizhou Gao, Jiajun Liang, Jian Wang, Chunxu Zhao, Weifeng Zhang, Lu Shi, Xin Li, Jiewei Yang, Hao Lu, Yunpeng Guan, Dongbo Gu, Chao Pei, Xiang Li, Zekun He (Tencent); Yong Jiang (Tsinghua University); Yachen Wang (Tencent)
Scribe: Yuntao Zhao (Xiamen University)
Introduction
As AI clouds increasingly serve diverse tenants for LLM training and inference, bare-metal GPU servers are attractive because they provide direct access to GPUs with low virtualization overhead. Unlike dedicated AI clusters, however, AI clouds must provide both network virtualization and tenant isolation while maintaining high network performance without requiring tenants to modify their communication frameworks.
Traditional clouds commonly use DPUs for network virtualization, but AI servers typically contain multiple GPUs and RNICs. Replacing every RNIC with a DPU increases total server power consumption by 6.5%–13.7%. Meanwhile, training, inference, and KV Cache traffic from different tenants have diverse communication patterns, and tenants may use their own communication libraries, making AI-cloud traffic more dynamic and unpredictable. To address these challenges, the paper presents Pegasus, a data center network designed for high-performance bare-metal AI clouds.
Key Idea and Contribution
Pegasus combines DPU-RNIC collaboration for network virtualization with RNIC-offloaded load balancing and congestion control, aiming to provide both tenant isolation and transparent high-performance communication.
(1) DPU-RNIC decoupled collaborative architecture
Instead of deploying a complete DPU for every GPU, Pegasus uses one DPU to manage multiple RNICs in each GPU server. The DPU handles network configuration, monitoring, and management, while tenant RDMA traffic continues to pass directly through the RNICs. Compared with replacing all RNICs with DPUs, this architecture saves 10.5% of power consumption per GPU server.
To support network virtualization, Pegasus maintains two-level flow tables on the DPU and RNICs for underlay-overlay IP translation, with transactional updates to preserve consistency. It also introduces a per-RNIC communication state machine to reduce unnecessary address validation and management traffic between the DPU and RNICs.
(2) High-performance transport offloading in the RNIC
To handle dynamic and unpredictable multi-tenant traffic, Pegasus offloads per-packet spraying and credit-based congestion control to the RNIC. Per-packet spraying provides fine-grained load balancing by generating ECMP entropy for individual packets, while credit-based congestion control proactively limits the amount of traffic injected into the network.
The paper further optimizes the RNIC implementation by moving order-preserving behavior for out-of-order packets from the sender to the receiver, avoiding an additional RTT, and by addressing credit jitter that can otherwise reduce throughput.
Evaluation
By February 2026, Pegasus had been deployed in production for two years, covering 8,000 GPUs across 1,000 GPU servers. Each server contains eight RNICs and one DPU, and the platform runs about 100 tenant tasks per day, with individual tasks using between 1 and 2,400 GPUs.
(1) LLM Training Performance
Pegasus is compared with the previous-generation AI cluster using the same models, parallelization strategies, and batch sizes. For a 70B dense model and a 7B MoE model trained on 32 GPUs, Pegasus improves training throughput by 7.5% and 4.8%, respectively. For LLaMA 7B training on 96 GPUs, the improvement reaches 53.2%.
(2) LLM Inference Performance
For a production inference workload using 16 GPUs located under different ToRs, Pegasus improves overall inference throughput by 3.7% and reduces the average completion time of user requests by 7.2% compared with the previous AI cluster.
Q&A
This paper did not have a Q&A session.
Personal thoughts
The most interesting aspect of Pegasus is that it considers the fundamental difference between a multi-tenant AI cloud and a dedicated AI cluster. Dedicated clusters can co-optimize the network and communication frameworks, while cloud tenants typically use their own training and communication stacks. Pegasus therefore provides network virtualization through DPU-RNIC collaboration and moves fine-grained load balancing and proactive congestion control into the RNIC, allowing the network to improve performance without requiring tenant-side modifications. However, the production experience also reveals new engineering challenges introduced by this architecture. Hardware flow-table hash collisions on the RNIC have caused tenant task interruptions, while DPU-RNIC management communication is still constrained by existing hardware interfaces.