Title: Spillway: Orchestrating DPU and Host into a Unified vSwitching Fabric
Authors: Xiaochong Jiang, Dian Fan, Yilong Lv, Naixuan Guan, Qiming Zhao, Sihan Fu, Ye Yang, Yu Zhou, Xuyang Ge, Denghui Wu, Yibin Shen, Guochun Hong, Zhipeng Zhang, Yijian Dong, Yiquan Chen, Shaoliang An, Zhixiong Guo, Yixin Xie, Yisong Qiao, Hongwei Ding, Haitao Jiang, Yimin Liu, Tao Shen, Baosheng Wang, Shize Zhang, Bowen Yang, Zikang Chen, Xiaomin Wu, Rong Wen, Yang Song, Jun Liang, Zhigang Zong, Xing Li, Chengkun Wei, Shunmin Zhu, Wenzhi Chen (Zhejiang University; Alibaba Cloud)
Review: Ying Yao(SNG first-year PhD student)
Introduction
Modern cloud data centers commonly employ Data Processing Units (DPUs) to offload virtual switches (vSwitches) and other infrastructure functions. The DPU’s hardware fast path handles established connections that hit the flow table, while the SoC’s software slow path manages flow table misses, policy computation, and state maintenance. This approach reduces host CPU overhead and ensures stable performance isolation for tenants.
However, a DPU’s compute, flow table, and memory capacities are determined by fixed hardware, and its upgrade cycle lags behind that of the host. Meanwhile, AI inference and Network Function Virtualization (NFV) workloads generate massive volumes of short-lived connections, bursty traffic, and fan-in/fan-out communication patterns. Data from Alibaba Cloud’s NFV clusters shows that over a 12-month period, bandwidth demand increased approximately sixfold, packet rates grew about fivefold, and concurrent sessions rose by roughly 1.2 times.
When the DPU reaches saturation first, the server cannot host additional network-intensive instances, even if CPU and memory resources remain available. These installed host resources—unable to be sold due to insufficient network capacity—effectively become “dark silicon.” Provisioning larger DPUs to handle peak traffic leads to low utilization; borrowing remote DPUs across nodes increases latency, east-west traffic, and state management overhead; and reverting to host-based software switching creates contention with tenants for PCIe, LLC, and memory bandwidth.
The paper introduces Spillway, a system that treats DPUs and hosts as two types of resources within a unified virtual switching fabric. Under normal conditions, DPUs handle the traffic; the system activates a host-side elastic data plane to process overflow traffic only when the DPU nears saturation and the host has safe, available idle resources, subsequently releasing those resources once the pressure subsides.
Key idea and contribution:
Spillway’s goal is not to replace the DPU with host software, but to enable the host to serve as an elastic overflow buffer for the DPU. The DPU retains the hardware fast path, while the DPU SoC and host vCPUs jointly form a hybrid slow path; a Unified Resource Scheduler (URSched) coordinates workloads across both sides based on load and resource utilization levels.
(1) Hosting the host-side vSwitch using an Elastic VM
Spillway encapsulates the host-side data plane within an Elastic VM rather than a kernel process or container. This approach leverages existing cloud platform mechanisms for VM creation and resource allocation while using hardware virtualization to define boundaries for CPU, cache, memory, and PCIe usage, thereby preventing interference with co-located tenants. To tenants and the upper-layer control plane, the DPU and host continue to appear as a unified vSwitch.
The Elastic VM communicates with the DPU via multiple sets of vNICs: a “flow vNIC” for rule installation and state synchronization, a “phy vNIC” for packet exchange with the DPU, and a “vm vNIC” for connecting to tenant VMs. Routine flow table updates are sent directly along the data plane to minimize latency, whereas flow table deletions—which require stricter correctness guarantees—are processed via a PCIe-based control channel. The system maintains two identical vNIC sets—one active and one “warm”—to facilitate failover and online upgrades through atomic rule redirection.
(2) On-demand startup and reclamation of Spillway
URSched monitors both DPU saturation levels and fragmented host resources. The system compares tenant traffic variance against the cluster-wide variance; it initiates Spillway preparation when tenant variance consistently exceeds the cluster average by approximately 25% or when DPU memory utilization reaches a threshold that could degrade performance.
The Elastic VM first synchronizes policies, flow tables, and hot state in “shadow mode” without altering the forwarding path. Once prepared, the DPU installs high-priority rules to redirect new connections and flow table misses to the Spillway, while existing connections already utilizing the hardware fast path continue to be processed by the DPU. Once the load stabilizes, the Spillway synchronizes its runtime state back to the DPU, removes redirection rules, drains in-flight packets, and releases host resources.
(3) Identifying stable flows suitable for hardware offloading
The host does not indefinitely retain all offloaded connections. Every 100 ms, Spillway extracts flow characteristics—such as bandwidth, packet rate, control packet ratio, and inter-packet jitter. After normalization, it employs a “Frequent Directions” sketch and streaming SVD to determine whether the traffic conforms to stable, recurring patterns.
The system further applies a Top-K strategy that jointly considers bandwidth gains and behavioral instability. Stable, high-bandwidth flows are re-installed into the DPU fast path, while bursty, complex, or highly variable flows remain in Spillway. This approach reserves scarce DPU table entries for flows that yield higher hardware efficiency, while offloading the burden of short-lived connections and state-related operations primarily to the host.
(4) Reducing PCIe round-trips with vATC
In a traditional receive path, packets that miss the fast path first enter the DPU SoC memory via DMA; after SoC processing, they are injected back into the fast path and finally sent to the target VM via PCIe—requiring at least two PCIe traversals. If Spillway and the tenant VM reside on the same host, these repeated returns to the DPU consume additional PCIe bandwidth shared between the host and the DPU.
To address this, Spillway implements a user-space Virtual Address Translation Cache (vATC) that maps device addresses to the tenant VM’s host physical addresses. Upon translation, Spillway can copy packets directly into the target VM’s memory, eliminating the PCIe round-trip otherwise required for re-injection into the DPU. Additionally, Elastic VM leverages “Shared Kernel State” to reuse the host’s memory management and interrupt descriptor tables, keeping cold-start memory usage below 50 MB.
Evaluation
The paper evaluates the system using both a controlled testbed and the Alibaba Cloud production environment. The testbed comprises hundreds of servers, each equipped with a CPU/FPGA hybrid DPU (2×100 Gbps); production results are derived from regions deploying Spillway on network-enhanced instances since mid-2024.
(1) Capacity Scaling and Burst Traffic
As the number of host vCPUs allocated to Spillway increases, the system’s CPS grows approximately linearly, achieving up to a 90% improvement over a pure-DPU architecture. In micro-burst experiments, the DPU’s processing capacity saturates at approximately 7.0 Mpps; when the input reaches 9.2 Mpps, the DPU continues to process about 7.2 Mpps, while Spillway absorbs the ~2.0 Mpps overflow, matching total throughput to the input rate without packet loss.
vATC is critical to performance: without it, Spillway’s PPS is only 48% of the DPU baseline, with 125% higher latency; with vATC enabled, PPS rises to 98% of the baseline, and latency drops by 37% compared to the disabled state. Regarding mode switching, the Elastic VM activates within 200 ms and rule injection takes approximately 10 ms, whereas the traditional cross-node failover used for comparison typically takes around 2 seconds.
(2) Resource Isolation
Under an overflow load of 2.0 Mpps, Spillway utilizes approximately 2.3 physical cores—representing 3.6% of a 64-core host. After sequentially enabling physical core pinning, NUMA/vATC, and CAT/MBA (cache and memory bandwidth control), the throughput of co-located tenants remains within 1% of the baseline (without Spillway), and Redis p99 latency increases by only 1.5%. If isolation targets cannot be met, URSched proactively reduces the overflow quota or reverts to the remote resource pool. (3) Production Deployment and Resource Gains
In practical deployment, Spillway increased the CPS-sensitive short-connection throughput of Nginx by approximately 25%, while Nginx long-connection performance remained largely unchanged. Redis QPS saw a decline of about 4.6%, attributed to some data traversing a longer software path. The authors consider this minor drop acceptable given scenarios where the DPU is already saturated and service interruptions or packet loss would otherwise occur.
In the production cluster, the available external compute capacity grew from approximately 120,000 vCPUs to about 180,000 vCPUs—a 50% increase. On representative servers, Spillway doubled CPS capacity using just seven physical cores while freeing up roughly 40–50 vCPUs that were previously unsellable; this equates to a ratio where investing one unit of compute resource recovers approximately three units of monetizable resource.
As of January 31, 2026, Spillway has been in continuous operation for over 1.5 years, deployed across more than 3,000 physical servers and supporting over 200,000 vCPUs; it achieves over 50% coverage in Availability Zones that offer network-enhanced instances.
QA
Q1: Why did Redis QPS experience a slight drop in the experiment?
A1: In a standard DPU architecture, data processed via the slow path can be sent directly from the DPU to the target VM; with Spillway enabled, however, some overflow data must first pass through the host-side Elastic VM before being forwarded to the target VM. Even though vATC eliminates one PCIe round-trip, this path remains longer than the direct DPU path, resulting in a Redis QPS decrease of approximately 4.6%. Nevertheless, only traffic exceeding the DPU’s processing capacity is routed to Spillway; this trade-off is acceptable when compared to the packet loss that would occur if the DPU became saturated.
Q2: Can VM placement adjustments—such as co-locating VMs with many short-lived connections alongside those with long-lived connections—be used to avoid the need for Spillway?
A2: While VM placement can help balance loads across hosts to some extent, the DPU handles the aggregate traffic of all tenants on a host; localized traffic spikes and state pressure can still saturate it. Spillway offers finer-grained differentiation at the flow level: stable, long-lived connections are ultimately offloaded to the DPU’s fast path, while bursty or short-lived connections remain on the host side. This eliminates the need to rely solely on coarse-grained, high-overhead VM migration to resolve the issue.
Q3: Is the Spillway VM dedicated to a specific tenant or VM?
A3: No. Spillway is multi-tenant; there is one Spillway VM per host (or per DPU) that handles overflow traffic for multiple tenant VMs associated with that DPU, rather than creating a separate data plane instance for each tenant.
Personal thoughts
Spillway’s most valuable insight is that DPU saturation is not synonymous with the exhaustion of the entire server’s resources. Instead of seeking remote DPUs across nodes, the system first leverages underutilized, fragmented CPU resources on the same host, effectively transforming fixed hardware network limits into elastically scalable capacity. It maintains a “DPU-first” approach—offloading only overflow traffic to the host—and seeks to preserve the advantages of the hardware fast path by migrating stable flows back to the DPU whenever possible.
The benefits of this approach are defined by specific workload boundaries. Short-lived connections, rule-insertion tasks, and state-intensive NFV workloads derive significant benefit; conversely, for long-lived connections and large flows—where forwarding bandwidth is the primary bottleneck—migrating to the host would merely increase PCIe and CPU overhead. RDMA training traffic, which typically bypasses the VPC vSwitch, falls outside Spillway’s scope. Suitability for the system also depends on factors such as the availability of idle host cores and the ability to strictly isolate PCIe and shared cache resources.
Production deployment experience demonstrates that unifying heterogeneous resources entails far more than simply forwarding packets to another processor; it requires the synchronization of policies, flow states, failure domains, and lifecycles. Spillway employs mechanisms such as pre-warming, shadow mode, sequential activation, and fail-open logic to prevent state loss during transitions—engineering strategies that were crucial to its successful deployment in a public cloud production environment.