[Paper Review] Adaptive Streaming in Interactive Multiview Video Systems

Xue Zhang, Laura Toni+3 more|arXiv (Cornell University)|Feb 25, 2018

Image and Video Quality Assessment36 references3 citations

TL;DR

This paper proposes an optimal adaptive streaming strategy for interactive multiview video systems that jointly selects the best set of camera views and their bitrates to maximize navigation quality under bandwidth constraints. Using dynamic programming and a greedy heuristic, the method reduces both coding and virtual synthesis artifacts, achieving significant improvements in PSNR and SSIM over baseline adaptation logic in both theoretical and realistic network conditions.

ABSTRACT

Multiview applications endow final users with the possibility to freely navigate within 3D scenes with minimum-delay. A real feeling of scene navigation is enabled by transmitting multiple high-quality camera views, which can be used to synthesize additional virtual views to offer a smooth navigation. However, when network resources are limited, not all camera views can be sent at high quality. It is therefore important, yet challenging, to find the right tradeoff between coding artifacts (reducing the quality of camera views) and virtual synthesis artifacts (reducing the number of camera views sent to users). To this aim, we propose an optimal transmission strategy for interactive multiview HTTP adaptive streaming (HAS). We propose a problem formulation to select the optimal set of camera views that the client requests for downloading, such that the navigation quality experienced by the user is optimized while the bandwidth constraints are satisfied. We show that our optimization problem is NP-hard, and we therefore develop an optimal solution based on the dynamic programming algorithm with polynomial time complexity. To further simplify the deployment, we present a suboptimal greedy algorithm with effective performance and lower complexity. The proposed controller is evaluated in theoretical and realistic settings characterized by realistic network statistics estimation, buffer management and server-side representation optimization. Simulation results show significant improvement in terms of navigation quality compared with alternative baseline multiview adaptation logic solutions.

Motivation & Objective

• Address the challenge of optimizing video quality in interactive multiview streaming under limited network resources.
• Balance trade-offs between coding artifacts (lower view quality) and virtual synthesis artifacts (fewer views transmitted).
• Develop an adaptation logic that considers both user interactivity and content characteristics for improved navigation quality.
• Ensure quality-of-service through bandwidth-aware client-side request selection and server-side representation optimization.

Proposed method

• Formulate the problem as an NP-hard optimization to select the optimal subset of camera views and their bitrates for download.
• Propose a dynamic programming-based optimal solution with polynomial-time complexity to solve the NP-hard problem.
• Design a suboptimal greedy algorithm with lower complexity and near-optimal performance for practical deployment.
• Integrate client-side adaptation logic with buffer management and realistic network trace estimation.
• Use server-side representation optimization based on joint view and bitrate selection, constrained by total storage per video (18 Mbps).
• Leverage depth-image-based rendering (DIBR) to synthesize virtual views, with quality dependent on anchor view quality and angular proximity.

Experimental results

Research questions

• RQ1 How can the optimal set of camera views and bitrates be selected to maximize navigation quality under bandwidth constraints?
• RQ2 What is the trade-off between coding artifacts (lower view quality) and virtual synthesis artifacts (fewer views) in multiview streaming?
• RQ3 How does user interactivity (navigation window dynamics) affect the optimal adaptation strategy?
• RQ4 Can a low-complexity heuristic achieve near-optimal performance while remaining deployable in real-world systems?

Key findings

• The proposed optimal dynamic programming solution achieves significant gains in navigation quality, with improvements of up to 0.05 in SSIM and 0.02 in PSNR over baseline methods.
• The suboptimal greedy algorithm achieves near-optimal performance with substantially reduced computational complexity, making it suitable for real-time deployment.
• In the low-interactivity scenario, the method maintains high-quality navigation across all users, with optimized representation sets varying per video and camera view.
• In the high-interactivity scenario, the method effectively handles non-uniform navigation patterns, with representation sets adapted to user-specific bandwidth and navigation variance (σ² = 10, 150, 3000).
• Buffer occupancy remains stable over time across all algorithms, with no re-buffering events, indicating effective bandwidth and buffer management.
• The joint optimization of server-side representation sets leads to diverse bitrate allocations per camera view, reflecting content-specific and user-specific characteristics.