Paper: Towards User-level QoE: Large-scale Practice in Personalized Optimization of Adaptive Video Streaming
Authors: Lianchen Jia, Chao Zhou, Chaoyang Li, Jiangchuan Liu, Lifeng Sun
Presenter: Hua Meng, The Hong Kong University of Science and Technology
Guest of Honor: Lianchen Jia, JD Technology
Q: How much computational overhead does the Monte Carlo simulation incur?
A: The Monte Carlo step is much more expensive than a single online decision — on the order of hundreds of times more computation according to our measurements. However, it is not executed for every decision. We only run the adjustment roughly once or twice per day, so its total online cost remains manageable.
Q: Why did you choose Monte Carlo simulation rather than a more sophisticated method?
A: The main reason is deployability. This work was intended for a real online system, so the method needed to be simple, understandable, and easy for engineers to diagnose and fix. Monte Carlo simulation is classical and straightforward, but that is actually an advantage in production. We deliberately preferred a method that engineers could understand and operate reliably rather than introducing unnecessary complexity.
Q: Why is there often a gap between sophisticated academic algorithms and the relatively simple methods used in industry?
A: I do not think simplicity means that industry and academia are fundamentally disconnected. For systems research, a method should be both useful and deployable. A complex or black-box approach may perform well, but once it is deployed, engineers also need to understand its behavior, observe failures, tune it, and repair it. Interpretability and operational simplicity therefore matter in addition to the final performance numbers. That is why I personally prefer work that is simple, useful, and can actually be deployed online.
Q: When the core method is simple but effective, how do you make the paper sufficiently insightful?
A: The most important part is not to make the mechanism look artificially complicated, but to clearly explain the important observation and why it matters. In this work, I spent substantial effort on the motivation and on showing what problems still remain in a mature domain after many years of research. The detailed mechanism itself can be described simply. A good systems paper can identify an important real problem, explain it clearly, provide an effective solution, and still leave room for future work to improve individual components.
Q: Which part of this research was the most challenging and time-consuming?
A: The hardest part was realizing that improving conventional video-system metrics did not necessarily improve the metric that mattered most to the business. We could improve video quality and reduce startup time, yet still see little improvement in user QoE. That forced us to change the way we thought about the problem: instead of optimizing lower-level quality metrics in isolation, we needed to focus directly on user experience. Making that conceptual shift was the most difficult part.
Q: Is the relative impact of video quality and smoothness on user exit rate stable across different application scenarios?
A: The exact numerical values will change across applications such as short-form and long-form video. However, we expect the general trend and relative relationship to remain similar. In other words, the detailed coefficients are scenario-dependent, but the qualitative trade-off should be relatively stable.
Q: Could large language models or autonomous agents eventually unify user-state understanding, QoE-risk assessment, and policy adaptation?
A: Large language models may have the capability to help with these tasks, especially because they can model users, time series, bandwidth, and other contextual information. The main obstacle is practical rather than conceptual: cost and response latency are critical in an online streaming system. Smaller models may offer a better trade-off, but any deployment would still need to balance model capability, inference cost, response time, and final system performance.
Q: Could a deep neural network replace the current hybrid approach entirely?
A: In principle, an end-to-end neural model could learn the relationship among video quality, smoothness, stall time, stall count, and other factors. However, those objectives interact in complicated ways, and stall-related metrics are especially important to QoE. We therefore use a hybrid approach rather than relying entirely on a DNN. A sufficiently sophisticated end-to-end model may be possible, but it would introduce additional complexity and deployment cost.
Q: Is it realistic for industry to release traces so that the research community can study this problem?
A: It is a useful direction. Some of the required information is essentially user response and network-level telemetry, so there may be opportunities to release appropriately processed traces. I would need to discuss this with industry partners to determine what can actually be shared. Another practical route is to collect controlled data from volunteers; prior QoE research has used this approach, and public QoE datasets already exist.
Q: Do different user groups respond differently to the system?
A: Yes. Users can be grouped in different ways, for example by system-level behavior or by demographic and geographic attributes such as age or city. These groupings may reveal different response patterns. This is a useful direction for further analysis, and more detailed segmentation studies could be conducted using large-scale platform data.
Q: Could the optimization objective be extended beyond watch time and stalling to include revenue or CDN cost?
A: Yes. This can naturally be formulated by expanding the reward or objective function. For an industrial system, objectives such as CDN cost, advertising revenue, or other business metrics may be important in addition to QoE. Joint optimization over these objectives is therefore a meaningful future direction.
Q: What did you learn from collaborating with industry and deploying the system?
A: The most important lesson is to keep the method as simple as possible. Engineers must be able to understand it, trust it, operate it, and fix it after the researchers are no longer involved. Industry partners also contributed something that is difficult to obtain in academia: large amounts of real online data. Learning how to extract useful signals and insights from production results was extremely valuable.
Q: What advice would you give to new graduate students who want to work on similar systems research?
A: Start by reading both classical and recent papers so that you understand the problem, why people care about it, and what the current hard points are. Once you understand the historical context and the present bottlenecks, the next research problem usually becomes much clearer. The process is incremental: first understand the field, then identify what remains genuinely difficult, and only then start designing your own solution.