Title: XFir: Accelerating New-Flow Setup on Host Servers of a Large Cloud Network
Authors: Shihan Lin (University of Michigan); Shunqiao Jiang, Liang Wang, Jian Wang, Chao Pei, Jian Zhao, Wenjun Wu, Kai Ren (Tencent); Lijun Zhuang, Qingmin Liu (JaguarMicro, Shenzhen); Heng Yu (Tsinghua University); Sirui Li (Tencent); Yibo Huang (University of Michigan); Yifei Zhu (Shanghai Jiao Tong University); Yunming Xiao (The Chinese University of Hong Kong, Shenzhen); Ang Chen (University of Michigan); Linghe Kong (Shanghai Jiao Tong University); Congcong Miao (National University of Singapore)
Introduction
In today’s cloud networks, host servers are increasingly relying on the “Sep-Path” architecture, where Data Processing Units (DPUs) are deployed to split packet processing into a programmable slow path (for new-flow setup) and a high-performance fast path (for established flows). While this paradigm balances throughput and flexibility, the rapid scaling of server capabilities—with hundreds of CPU cores and hundreds of Gbps of network bandwidth—has exposed a critical bottleneck: the software slow path, executed on a DPU’s CPU, struggles to keep up with the rising rate of new connections, especially for workloads dominated by short-lived flows such as instant messaging and e-commerce promotions. For example, a large cloud customer requires over 2K new connections per second (CPS) per CPU core, translating to more than 700K CPS on a 350-core server, far exceeding the 160K CPS that current DPUs can sustain. Existing solutions fall short because running flow setup on the host’s CPU consumes precious resources meant for tenant workloads, attaching multiple DPUs per server is prohibitively expensive at hyperscale, and FPGAs or programmable ASICs lack the flexibility or on-chip capacity to accommodate the complex, evolving table lookups required for host-server new-flow setup.
Key idea and contribution
To address this bottleneck, the authors propose XFir, the first hardware-accelerated new-flow setup system for cloud host servers. XFir leverages a next-generation DPU, Xline, which integrates a fixed-function ASIC, a general-purpose CPU, and a novel Cloud Network co-Processor (CNP)—a lightweight programmable pipeline combined with a RISC-V coprocessor. The key insight is that the CNP offers sufficient flexibility to implement the entire slow-path logic while achieving substantially higher throughput than CPU-based implementations. The authors redesign the datapath and table layout to fully exploit the CNP’s processing units, migrating the slow-path logic from the DPU’s CPU to the CNP and carefully partitioning tables between the pipeline and the RISC-V cores based on their size, statefulness, and access patterns.
XFir introduces three core design components: (1) a restructured datapath and table layout that places flow setup, global ACL, and per-NIC ACL tables in the pipeline for fast TCAM-based lookups, while routing, VNIC, and stateful Security Group tables are handled by the RISC-V coprocessor; (2) an optimized Longest Prefix Match (LPM) algorithm that combines hashing with tries, exploiting the skewed prefix-length distribution in cloud networks (most prefixes are between 17 and 24 bits) to reduce DRAM accesses and lookup latency; and (3) CPU-CNP collaboration mechanisms, including CPU interposition for observability, throughput aggregation via parallel flow processing, and an escape path for reliability during hardware failures. Together, these innovations enable XFir to deliver high CPS throughput while preserving the flexibility required for continuous feature evolution and maintaining cost-effectiveness for per-host deployment.
Evaluation
The authors evaluate XFir against two baselines: SW (a software slow-path implementation on Xline’s CPU) and Fornax (a state-of-the-art FPGA-based DPU solution). Their experiments show that XFir achieves over 776K new-flow CPS on a single host server, representing a 4.8× improvement over Fornax and a 4.4× improvement over SW, with a median slow-path latency of 11.7 μs—69.2% lower than Fornax. The optimized LPM algorithm reduces DRAM accesses to at most seven for the most common prefix lengths (17–24 bits), compared to traditional trie-based approaches that require significantly more accesses for longer prefixes. An ablation study demonstrates that both the version control mechanism and the hybrid table layout contribute to the CPS gains, with the CNP’s RISC-V coprocessor providing most of the acceleration due to its specialized network instructions. Cost analysis shows that XFir’s per-host deployment cost, normalized to 100 Gbps of bandwidth, is 0.25, lower than both Fornax (0.46) and Triton (0.50), making it economically viable for hyperscale cloud deployments. This result is significant because it demonstrates that hardware acceleration of new-flow setup can keep pace with the rapidly increasing server core counts and network bandwidth, without sacrificing the flexibility required for cloud providers to continuously evolve their networking logic or incurring prohibitive cost at scale.
Q1: What would be the ideal SmartNIC architecture and its key ingredients if you were designing it from first principles without any hardware constraints?
A1: The ideal SmartNIC would offer both complete programmability and line-rate performance. Since this is physically impossible due to hardware design constraints, real-world systems must always be fine-tuned and co-designed based on currently available hardware.
Q2: What kind of traffic was used during the evaluation, how long did the connections last, and how does the connection duration affect the system’s throughput?
A2: The evaluation used synthetic TCP traffic generated between two DPU-equipped machines to overload the receiver. Under a heavy traffic load lasting for one minute, the system sustained a throughput of around 380 Gbps, which is extremely close to the DPU’s 400 Gbps line rate.
Q3: How is the workload divided between the programmable pipeline and the co-processor in the CMP, how do they exchange data, and could the system function using the RISC-V co-processor alone?
A3: Workloads are divided by mapping different tables to different processing paths, and data is exchanged directly via shared on-chip TCAM and SRAM. Moving all tables to the RISC-V co-processor alone is inefficient and would degrade the connection-per-second (CPS) performance by approximately 90,000.
Q4: What is the fundamental bottleneck of the slow path in traditional DPUs, and how does XFir optimize and overcome this bottleneck?
A4: The main bottlenecks are slow CPU cycles, high-latency DRAM accesses, and DMA overheads. XFir eliminates these by bypassing the CPU entirely and running the slow path on the CMP using assembly-programmed hardware, keeping all components on-chip to avoid DMA and DRAM latency.
Personal thoughts
XFir presents a compelling system design that effectively tackles a real and growing bottleneck in cloud networking—the slow path of new-flow setup. The paper’s strength lies in its practical, deployment-driven approach: the authors have not only identified a critical problem but have also built a system that is already in canary testing on about 1,000 host servers in their production cloud. The software-hardware co-design is thoughtful, particularly the careful partitioning of tables between the CNP pipeline and RISC-V coprocessor, the optimized LPM algorithm tailored to cloud prefix distributions, and the CPU-CNP collaboration mechanisms that enhance both performance and reliability. The evaluation is comprehensive and the performance gains over prior work are substantial, making a strong case for the viability of this approach.
However, one limitation is the tight coupling of XFir’s design to the specific Xline DPU architecture. While the authors discuss generalizable principles, the CPS improvements heavily rely on the CNP’s unique capabilities, which may not be available on other DPU platforms. This raises questions about the broader applicability of the approach—would similar gains be achievable with a different hardware architecture, or does it require custom silicon? Additionally, the paper’s focus on IPv4 and the specific table lookup workflow of the authors’ cloud environment may limit its direct applicability to other cloud providers with different networking stacks or IPv6 adoption. That said, the design rationales, such as the FST and version control, are likely transferable, and the paper opens up several interesting research directions: (1) how to further scale CPS to 2M or beyond as server core counts and bandwidth continue to grow; (2) whether similar co-design principles can be applied to other stateful network functions beyond flow setup; and (3) how to enable more flexible programmability on the CNP without sacrificing performance, potentially through domain-specific languages or higher-level abstractions. Overall, XFir is an impressive piece of systems research that demonstrates the untapped potential of next-generation DPUs with integrated coprocessors, and it sets a new bar for host-server new-flow setup performance.