Title: Contiguous Zero-Copy for Encrypted Transport Protocols
Authors: Florentin Rochet (UNamur, Belgium)
Scribe: Yuntao Zhao (Xiamen University)
Introduction
Modern transport protocols increasingly encrypt both application data and protocol control information. QUIC, for example, may mix application data with multiple control frames inside the same encrypted payload. With conventional wire formats, the receiver must decrypt the packet, parse control information, locate data fragments, and then copy these fragments into contiguous memory before delivering them to the upper layer. As a result, even when cryptographic operations are hardware-accelerated, the protocol wire format itself can introduce additional memory-copy and CPU overhead.
To address this issue, the paper proposes Reverso. Its key observation is that contiguous zero-copy processing on the receiver is not purely an implementation issue but also depends on the protocol wire image. By rearranging data and control fields and changing how decrypted content is parsed, Reverso enables application data to be placed directly into a contiguous upper-layer buffer without modifying the existing symmetric cryptography interface. The paper applies this methodology to QUIC as QUIC VReverso and implements a corresponding QUIC stack called quiceh.
Key Idea and Contribution
(1) Enabling contiguous zero-copy through wire-format redesign
Reverso introduces two main design principles. First, protocol control fields are reversed so that the receiver can process decrypted control information from right to left. Second, application data is placed first in the encrypted content, followed by its associated control information. This allows subsequent packets to be decrypted directly after previously received data, using the write operation already performed by the cryptographic primitive to construct contiguous application data without an additional reassembly copy.
Instead of relaxing the atomic AEAD interface to obtain this optimization, Reverso changes the protocol layout while preserving the existing cryptographic interface, avoiding the security concerns associated with more flexible partial-decryption APIs.
(2) Applying Reverso to QUIC
The paper applies these principles to QUIC VReverso. Its main changes include placing the Stream Frame first in the encrypted payload, reversing field ordering within QUIC frames, and extending the short header with information such as the Stream ID and offset so that the receiver can determine the destination stream buffer before payload decryption. Existing QUIC features such as connection migration, congestion control, and multiple streams are preserved.
The authors implement quiceh based on Cloudflare’s quiche. It supports both QUIC V1 and QUIC VReverso and provides a contiguous zero-copy receive API that allows applications to access decrypted stream data directly without additional internal reassembly copies.
Evaluation
The evaluation mainly compares QUIC V1 in quiche with QUIC VReverso in quiceh and examines whether the optimization also benefits HTTP/3.
(1) QUIC and HTTP/3 Processing Efficiency
Microbenchmarks show that VReverso eliminates two full memory copies of application data on the receive path and improves QUIC packet-processing throughput by approximately 30% across all three tested processors.
For HTTP/3 processing of a 2.5 MB page distributed across 80 request/response streams, throughput improves by approximately 38% on Xeon Gold and Intel i7 processors and by approximately 26% on the tested Ryzen processor.
(2) Real-World Experiments
Because packet reordering prevents the receiver from always obtaining the full contiguous zero-copy benefit, the paper also measures ordering across Internet paths. In HTTP/3 transfers among vantage points in North America, Europe, Singapore, and Australia, more than 99.5% of packets are ordered on most paths. An anomaly at the Belgian vantage point reduces the ordered fraction to approximately 89% to 98.4%, which the authors trace to a nearby firewall.
In a 1 Gbps HTTP/3 download experiment, QUIC VReverso further reduces receiver CPU utilization, and its benefit is cumulative with UDP batching optimizations.
Q&A
This paper did not have a Q&A session.
Personal thoughts
By changing how data and control fields are arranged on the wire, Reverso allows the memory write already performed during AEAD decryption to directly construct the contiguous data representation required by the upper layer. This shows that protocol wire formats can directly determine which zero-copy optimizations are possible in implementations. However, the practical benefit of Reverso depends on both the QUIC implementation and packet ordering. The current evaluation mainly focuses on quiche and quiceh, while other QUIC stacks use different architectures and APIs and may therefore obtain different levels of improvement.