[Paper Review] Spatial and Temporal Consistency-Aware Dynamic Adaptive Streaming for 360-Degree Videos

Hui Yuan, Shiyun Zhao+3 more|arXiv (Cornell University)|Dec 20, 2019

Image and Video Quality Assessment51 references4 citations

TL;DR

This paper proposes a spatial and temporal consistency-aware dynamic adaptive streaming framework for 360-degree videos that improves quality of experience (QoE) by jointly optimizing bitrate allocation, field-of-view (FoV) prediction, and tile prioritization. Using a Gaussian model for FoV prediction and a Zipf-based priority model for tile downloading, the method employs a two-stage optimization to balance FoV quality, spatial consistency, and temporal smoothness, achieving superior QoE compared to state-of-the-art methods in both controlled and real network environments.

ABSTRACT

The 360-degree video allows users to enjoy the whole scene by interactively switching viewports. However, the huge data volume of the 360-degree video limits its remote applications via network. To provide high quality of experience (QoE) for remote web users, this paper presents a tile-based adaptive streaming method for 360-degree videos. First, we propose a simple yet effective rate adaptation algorithm to determine the requested bitrate for downloading the current video segment by considering the balance between the buffer length and video quality. Then, we propose to use a Gaussian model to predict the field of view at the beginning of each requested video segment. To deal with the circumstance that the view angle is switched during the display of a video segment, we propose to download all the tiles in the 360-degree video with different priorities based on a Zipf model. Finally, in order to allocate bitrates for all the tiles, a two-stage optimization algorithm is proposed to preserve the quality of tiles in FoV and guarantee the spatial and temporal smoothness. Experimental results demonstrate the effectiveness and advantage of the proposed method compared with the state-of-the-art methods. That is, our method preserves both the quality and the smoothness of tiles in FoV, thus providing the best QoE for users.

Motivation & Objective

• To address the challenge of delivering high-quality 360-degree video over dynamic network conditions with limited bandwidth.
• To reduce viewport-switching delay and maintain visual consistency when users rapidly change their viewing angle.
• To optimize bitrate allocation across tiles to preserve both spatial and temporal smoothness in the field of view (FoV).
• To improve user quality of experience (QoE) by integrating buffer management, FoV prediction, and adaptive tile downloading strategies.
• To develop a robust, end-to-end framework that outperforms existing methods in diverse network and viewing scenarios.

Proposed method

• A novel rate adaptation algorithm balances buffer length and video quality to determine the requested bitrate for each video segment.
• A Gaussian model predicts the field of view (FoV) at the start of each video segment based on user head movement trends.
• All tiles in the 360-degree video are downloaded with different priorities using a Zipf-based model to account for potential viewport switching during playback.
• A two-stage optimization algorithm allocates bitrates to tiles: first coarse allocation to ensure FoV quality, then fine-tuning to preserve spatial and temporal consistency.
• The method integrates DASH-based streaming with tile-level encoding and dynamic bitrate adaptation to support adaptive quality delivery.
• The framework ensures smooth transitions and high visual quality even during sudden viewport changes by preloading relevant tiles with prioritized bandwidth.

Experimental results

Research questions

• RQ1 How can bitrate adaptation be optimized to balance buffer stability and video quality in 360-degree video streaming?
• RQ2 To what extent can FoV prediction reduce viewport-switching delay and improve QoE in dynamic viewing scenarios?
• RQ3 How can tile prioritization based on user viewing probability improve bandwidth efficiency and visual consistency?
• RQ4 What is the impact of spatial and temporal smoothness on perceived video quality in 360-degree video delivery?
• RQ5 How does the proposed two-stage optimization outperform existing methods in maintaining FoV quality and consistency under varying network and user behavior?

Key findings

• The proposed method achieves the lowest FoV PSNR standard deviation (Std) and temporal difference, indicating superior temporal smoothness.
• The proposed method records the smallest F-value (a metric combining quality and smoothness) across all tested scenarios, including 5%, 10%, and 20% sudden viewport-switching probabilities.
• In real network conditions, the proposed method achieves the highest weighted PSNR and the best spatial and temporal smoothness, as confirmed by visual and quantitative analysis.
• The proposed method delivers the highest QoE score under the QoE metric (16), with a 15-second buffer reference, outperforming AdpaA, PD, and AA methods.
• Even when sudden viewport switching occurs, the proposed method maintains the lowest F-value and highest QoE, demonstrating robustness across diverse video content like AerialCity, DrivingInCountry, and PoleVault.
• The method achieves the best trade-off between FoV quality, spatial consistency, and temporal smoothness, with the PD method performing poorly under sudden switching due to lack of preloaded tiles.