Title: CacheFlare: Optimizing Cold Content Performance in CDNs
Authors: Tiansheng Zhang, YuLing Chen, Ahmed Kamal, Liang Zhou, Jianfeng Tang, Huapeng Zhou, Thilan Ganegedara, Yuhan Guo, Sanjay Sane, Ben Vallis (Meta Platforms, Inc.); Theophilus A. Benson (Carnegie Mellon University); Ying Zhang (Meta Platforms, Inc.)
Introduction
CDNs normally improve edge hit rates through repeated access, but images and videos shared in direct messages circulate only among individuals or small groups, and much of this content is accessed once or only a few times. The first download therefore often misses at the edge and fetches the object from an origin datacenter. Meta’s production data shows that such cold content accounts for a large portion of direct-messaging traffic, lowers edge hit rates, and creates longer delivery paths for users far from origin datacenters. CacheFlare uses Upload Caching and Cache Priming according to whether content requires datacenter-side transcoding, sanitization, or other processing, placing the object in the L2 edge cache before the user’s request arrives.
Key idea and contribution:
CacheFlare divides cold content into two categories and selects a different proactive caching point within the existing multi-tier CDN. Content that requires no post-processing can be cached directly on the upload path, while content that must first be processed in a datacenter is pushed to the target edge cluster after its CDN URL is generated.
- Upload Caching: When a sender uploads content that requires no post-processing, the L2 node uses a write-through design to store the data in both its local cache and the origin. After the final CDN URL is returned, a redirect key maps the persistent URL to the cache key used during upload, allowing the receiver’s first download to hit the L2 cache. When both parties are online, Express Path can share the upload URL earlier, enabling the receiver to read already cached pieces while the upload is still in progress (Figure 8).
- Cache Priming: For content that requires transcoding or sanitization, the application sends the targeted CDN URL produced by URLGen to the Cache Priming Service. CPS identifies the target cluster, maps an L1 target to the corresponding L2 cluster, retrieves the processed object from the datacenter, and writes it into the L2 cache through an HTTP POST. The system records recent priming requests in Memcache and uses Request Coalescing to combine repeated writes of the same object to the same edge cluster, reducing over-priming and backbone traffic (Figure 9).
Evaluation
After Upload Caching is deployed, the byte-level cache hit rate for eligible traffic increases by about 7 percentage points, datacenter-to-PoP bandwidth falls by about 18%, and P50 time to last byte falls by about 17%. In the same application, Express Path further reduces P90 download latency by about 60%.
In the direct-messaging image experiment, Cache Priming raises the overall image hit rate from 30% to 41% and the edge hit rate from 24% to 70%. Average CDN time to first header byte falls by 40% and P50 falls by 92%. User image-render latency decreases by 9.3%, the percentage of poor renders decreases by 4.7%, and incremental backbone usage is 0.1% (Table 2).
At first, Cache Priming repeatedly primes the same content, producing traffic spikes and unnecessary origin fetches. After Request Coalescing is added, CPS request rate and edge-to-origin backbone usage both decrease by 50-60%, and the maximum edge-cache SET request rate falls by about 90%, substantially reducing the infrastructure pressure caused by proactive caching (Figure 12). These results matter because they demonstrate that application lifecycle signals can make the first request for intrinsically cold content benefit from edge caching while keeping the added infrastructure cost under control.
Q&A
Q1: How did you decide the thresholds for cold and hot content? You define cold content as content accessed 2-10 times and hot content as content accessed more than 10 times. Were these thresholds determined statistically, or were they based on observed user behavior and cache performance?
A1: One-hit wonders speak for themselves. For cold content, the threshold is more heuristic. Although we define hot content as content accessed more than 10 times, those accesses may be distributed across many edge clusters rather than concentrated in a single cluster. If the 10 accesses are in one cluster, the cache hit rate can still be decent; if they are spread across clusters, the cache hit rate may not be good.
Q2: We are a research community, so we are interested in hard and important problems. Based on this system, what open problems do you think would be interesting to tackle?
A2: One open problem is excessive backbone usage. We are still investigating why it happens. It may be related to the prefetching logic used by client applications. For example, Instagram and Facebook prefetch a lot of content, which generates many CDN URLs and indirectly increases backbone usage during cache priming. We are also looking at how to cache content better across edge clusters and how to enable cross-edge-cluster sharing, so that we can improve cache efficiency and reduce serving latency.
Q3: I noticed that the paper says video is not supported yet. Does video introduce any interesting differences or challenges?
A3: We recently applied Cache Priming to video content during the first half of the year and got some results. They are not included in the paper. In the application surface where we applied Cache Priming, the average video was about two or three times larger than the average image we served. I cannot discuss the specific application.
Personal thoughts
CacheFlare’s treatment of cold content is noteworthy. Traditional cache policies usually decide whether an object is worth retaining according to historical access frequency, while direct-messaging workflows provide a clear signal of future access when the content is uploaded or its CDN URL is generated. The paper uses these workflow events to change the caching point, allowing a first download that would receive little benefit from reactive caching to hit an edge node. The design gains its effectiveness from connecting the content lifecycle with the CDN data path rather than introducing a complex prediction model.
The deployment experience also shows that proactive placement turns a hit-rate problem into a resource and reliability problem. Repeated priming created traffic spikes and exposed a kernel defect on edge servers, making Request Coalescing necessary. Upload Caching remains constrained by sender-receiver location and content-processing requirements, while Cache Priming depends on targetable URLs and must control bandwidth spent on objects that users never request. These constraints become more important when expanding the system to larger video objects.