[论文解读] Toward One-Second Latency: Evolution of Live Media Streaming

Abdelhak Bentaleb, May Lim+4 more|arXiv (Cornell University)|Oct 5, 2023

Image and Video Quality Assessment被引用 9

一句话总结

本论文调研了实时媒体流媒体向低延迟交付的发展，详细介绍延迟来源、架构、协议，以及 LL-DASH 和 LL-HLS 等扩展，并讨论 Twitch 的挑战及未来研究方向。

ABSTRACT

This survey presents the evolution of live media streaming and the technological developments behind today's IP-based low-latency live streaming systems. Live streaming primarily involves capturing, encoding, packaging and delivering real-time events such as live sports, live news, personal broadcasts and surveillance videos. Live streaming also involves concurrent streaming of linear TV programming off the satellite, cable, over-the-air or IPTV broadcast, where the programming is not necessarily a real-time event. The survey starts with a discussion on the latency and latency continuum in streaming applications. Then, it lays out the existing live streaming workflows and protocols, followed by an in-depth analysis of the latency sources in these workflows and protocols. The survey continues with the technology enablers, low-latency extensions for the popular HTTP adaptive streaming methods and enhancements for robust low-latency playback. An entire section is dedicated to the detailed summary and findings of Twitch's grand challenge on low-latency live streaming. The survey concludes with a discussion of ongoing research problems in this space. We expect this survey to be the one-stop reference for those who would like to learn how low-latency live streaming has evolved and works today, and what further developments could happen in the future.

研究动机与目标

• 提供对实时流和低延迟实时流组件及其对端到端延迟影响的全面解剖。
• 描述能实现鲁棒、低延迟播放的前沿技术及扩展。
• 分析在媒体内容准备、传输和消费阶段的延迟来源。
• 总结来自行业挑战（如 Twitch）的关键发现，并概述尚待解决的研究问题。

提出的方法

• 将实时流架构分为媒体贡献、采集和分发阶段，并将协议映射到每个阶段。
• 通过将端到端延迟分解为内容准备、传输和消费延迟来分析。
• 以 CMAF、HTTP/1.1/HTTP/2/HTTP/3（QUIC）为示例，讨论分块编码/打包与分块传递。
• 评审 DASH 的低延迟扩展（LL-DASH）和 HLS 的低延迟扩展（LL-HLS）及其运行影响。
• 通过实时协议和 IETF MOQ 讨论，总结交互性和实时性方面。
• 评估 Twitch 的 grand challenge，以展示端到端 LLL 流式原型设计与测试流程的实际应用。

Figure 1 : Continuum of different E2E latency values. Latency values are not to scale.

实验结果

研究问题

• RQ1 当前实时流工作流中的主要延迟组成部分有哪些，它们如何累积成端到端延迟？
• RQ2 DASH 与 HLS 的低延迟扩展在实际部署中如何降低端到端延迟？
• RQ3 哪些架构与协议选择能够实现稳健的实时流低于 10 秒的延迟？
• RQ4 从 Twitch 的 grand challenge 中可以得到哪些用于部署实时、低延迟流的经验？
• RQ5 低延迟实时流领域尚存的研究问题和未来方向是什么？

主要发现

• LLL 流媒体的端到端延迟目标是低于 10 秒，理想地约五秒。
• LL-DASH 和 LL-HLS 通过分块编码和传送扩展 HAS 以降低延迟。
• CMAF 与分块传输编码是实现 DASH/HLS 低延迟传送的核心。
• HTTP/3（QUIC）和服务器推送特性通过减少建立和获取时间来影响延迟；H2/H3 服务器推送在 LLL 情境中已显示潜力。
• 低延迟分段、自适应码率与缓冲管理策略必须与编码器前瞻和 ABR 阶梯共同优化，以在 QoE 与延迟之间取得平衡。
• Twitch 的 grand challenge 为端到端 LLL 流式原型设计提供最佳实践洞见。

Figure 2 : A typical E2E live streaming architecture. The thickness of the arrows denotes the bitrate for that particular media stream (not to scale). There are usually multiple media streams (audio, video, metadata, subtitles, etc. ) acquired from the source (not shown).