[Paper Review] Markov Decision Policies for Dynamic Video Delivery in Wireless Caching Networks

Minseok Choi, Albert No+2 more|arXiv (Cornell University)|Feb 28, 2019

Caching and Content Delivery33 references4 citations

TL;DR

This paper proposes a dynamic video delivery policy in wireless caching networks that jointly optimizes caching node association, video quality, and chunk transmission volume using Lyapunov optimization and Markov decision processes. It achieves a balanced tradeoff between high time-average video quality and low playback delay, outperforming existing schemes in both metrics while enabling tunable performance via the system parameter $V$.

ABSTRACT

This paper proposes a video delivery strategy for dynamic streaming services which maximizes time-average streaming quality under a playback delay constraint in wireless caching networks. The network where popular videos encoded by scalable video coding are already stored in randomly distributed caching nodes is considered under adaptive video streaming concepts, and distance-based interference management is investigated in this paper. In this network model, a streaming user makes delay-constrained decisions depending on stochastic network states: 1) caching node for video delivery, 2) video quality, and 3) the quantity of video chunks to receive. Since wireless link activation for video delivery may introduce delays, different timescales for updating caching node association, video quality adaptation, and chunk amounts are considered. After associating with a caching node for video delivery, the streaming user chooses combinations of quality and chunk amounts in the small timescale. The dynamic decision making process for video quality and chunk amounts at each slot is modeled using Markov decision process, and the caching node decision is made based on the framework of Lyapunov optimization. Our intensive simulations verify that the proposed video delivery algorithm works reliably and also can control the tradeoff between video quality and playback latency.

Motivation & Objective

• Address the challenge of dynamic video delivery in wireless caching networks where users experience varying network conditions and mobility.
• Optimize three interdependent decisions: caching node association, video quality level, and number of video chunks to transmit, under delay constraints.
• Maximize time-average video quality while ensuring playback delay remains acceptably low in a stochastic, time-varying network environment.
• Introduce a multi-timescale decision framework where node association is updated slowly, while quality and chunk size are adapted rapidly using MDP.
• Enable tunable tradeoff between video quality and playback delay through the Lyapunov optimization parameter $V$.

Proposed method

• Model the fast-timescale decisions (video quality and chunk amount) using a Markov decision process (MDP) to capture state-dependent actions.
• Use frame-based Lyapunov optimization to manage the slow-timescale caching node association decision, ensuring stability and delay constraints.
• Formulate the problem as a drift-plus-penalty minimization to balance video quality maximization and delay avoidance.
• Apply dynamic programming to solve the MDP component, determining optimal quality-chunk combinations per time slot based on current channel and queue states.
• Integrate the MDP policy with the Lyapunov-based node association policy, enabling joint optimization across different timescales.
• Leverage scalable video coding (SVC) to allow different quality layers of the same video to be cached and delivered independently.

Experimental results

Research questions

• RQ1 How can caching node association, video quality, and chunk transmission volume be jointly optimized in a dynamic wireless caching network with delay constraints?
• RQ2 What is the impact of different timescales on the performance of video delivery policies in mobile, stochastic environments?
• RQ3 Can a Lyapunov optimization framework effectively balance video quality and playback delay while maintaining stability?
• RQ4 How does the proposed policy compare to static or heuristic policies like 'Strongest', 'Highest-Quality', and 'One-Step' in terms of quality and delay tradeoffs?
• RQ5 To what extent can the system parameter $V$ be used to tune the tradeoff between video quality and playback delay?

Key findings

• The proposed policy achieves significantly higher average video quality than the 'Strongest' and 'Highest-Quality' schemes, while maintaining a playback delay occurrence rate comparable to 'Strongest'.
• The 'One-Step' scheme achieves higher quality than the proposed method at low $V$ but suffers from poor delay performance, indicating its inability to control delay effectively.
• As $V$ increases, the proposed scheme improves video quality with a minimal increase in delay incidence, demonstrating effective tunability of the quality-delay tradeoff.
• The proposed scheme maintains robust performance across varying SINR levels, outperforming 'Highest-Quality' which degrades significantly under high interference.
• The 'Strongest' policy shows stable delay performance across SINR levels due to strong channel selection, but the proposed method achieves better quality under the same delay constraints.
• The performance rankings among schemes remain consistent across different metrics and parameter settings, confirming the reliability and superiority of the proposed approach.