Title: APNet Rising Star: Networking for AI-Native Data
Speaker: Junchen Jiang (University of Chicago)
Session: Best of Networking
Scribe: Letian Zhu (Xiamen University)
Introduction
Rather than presenting individual papers, Junchen Jiang used this talk to discuss a broader shift from human-native data to AI-native data. He argued that as large-scale AI inference becomes increasingly common, models are generating and consuming large amounts of internal data that is not designed for humans. KVCache is an important example: unlike text, images, or video, it is not human-readable but can be directly consumed by models and captures information from previously processed tokens. Jiang therefore described KVCache as one of the first widely adopted forms of AI-native data.
He also emphasized the gap between academic KVCache research and industrial deployment. Although research has explored compression, cross-model sharing, and other semantic-aware optimizations, production systems historically lacked common infrastructure and interfaces for storing and manipulating KVCache. This motivated LMCache, an open-source project designed first around broad ecosystem integration and ultimately intended to provide a bridge through which research ideas can enter production systems.
Questions and opinions:
Key Questions：
-
Why should KVCache be viewed as AI-native data rather than simply a cache?
-
Why are many KVCache research ideas difficult to deploy in industry?
-
What systems and networking principles are needed for AI-native data?
-
Can KVCache become an interface for influencing model behavior?
Jiang argued that KVCache is more than a performance optimization because it contains information related to a model’s internal processing of previous tokens. Understanding this structure enables techniques such as compression and cross-model sharing, while modifying KVCache may also affect model attention and outputs.
A major obstacle is that production systems tend to optimize around existing assumptions. Research techniques that inspect or manipulate KVCache require interfaces that many inference systems do not expose. LMCache therefore prioritized compatibility with inference engines, storage systems, and hardware platforms before introducing more sophisticated research interfaces.
More broadly, Jiang argued that AI-native data raises fundamental systems questions involving the data plane, control plane, quality of service, global routing, fault tolerance, security, and privacy. He encouraged the networking community to move beyond individual KVCache optimizations and develop general design principles for this emerging infrastructure.
Discussion
Q1: Why is there a gap between KVCache research and industrial deployment?
A1: Jiang argued that once production systems are deployed and heavily optimized, changing their fundamental assumptions becomes difficult. Early inference systems did not provide suitable mechanisms for storing or manipulating KVCache, limiting the deployment of more advanced research ideas. A widely adopted open-source infrastructure such as LMCache could provide an entry point for researchers to introduce new mechanisms into production systems.
Q2: Can KVCache help humans influence or control the behavior of trained models?
A2: Jiang suggested that KVCache could become an interface between inference-system operators and model behavior. Inference engineers can access both runtime prompts and KVCache without modifying model weights, creating opportunities to influence model behavior at inference time. However, he emphasized that this is still an emerging direction and currently lacks sufficiently principled approaches.
Personal thoughts
I found the framing of KVCache as AI-native data more interesting than viewing it only as an inference optimization. The talk also highlighted an important systems problem: research ideas need practical abstractions and interfaces before they can influence production systems. More broadly, AI-native data may create new research opportunities in networking, storage, reliability, security, and distributed AI systems.