[Paper Review] QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks

Rajarshi Bhattacharyya, Archana Bura+6 more|arXiv (Cornell University)|Jan 4, 2019

Image and Video Quality Assessment4 citations

TL;DR

QFlow proposes a reinforcement learning (RL)-based system for dynamic video streaming quality optimization in wireless networks by adaptively assigning clients to priority queues based on real-time QoE feedback. Using both model-free and model-based RL, QFlow achieves over 25% higher QoE than baseline methods and maintains a perfect QoE score (5/5) over 85% of the time under high load.

ABSTRACT

Wireless Internet access has brought legions of heterogeneous applications all sharing the same resources. However, current wireless edge networks that cater to worst or average case performance lack the agility to best serve these diverse sessions. Simultaneously, software reconfigurable infrastructure has become increasingly mainstream to the point that dynamic per packet and per flow decisions are possible at multiple layers of the communications stack. Exploiting such reconfigurability requires the design of a system that can enable a configuration, measure the impact on the application performance (Quality of Experience), and adaptively select a new configuration. Effectively, this feedback loop is a Markov Decision Process whose parameters are unknown. The goal of this work is to design, develop and demonstrate QFlow that instantiates this feedback loop as an application of reinforcement learning (RL). Our context is that of reconfigurable (priority) queueing, and we use the popular application of video streaming as our use case. We develop both model-free and model-based RL approaches that are tailored to the problem of determining which clients should be assigned to which queue at each decision period. Through experimental validation, we show how the RL-based control policies on QFlow are able to schedule the right clients for prioritization in a high-load scenario to outperform the status quo, as well as the best known solutions with over 25% improvement in QoE, and a perfect QoE score of 5 over 85% of the time.

Motivation & Objective

• To address the lack of agility in current wireless edge networks that prioritize worst-case or average-case performance over diverse, dynamic application demands.
• To enable real-time, per-flow reconfiguration across multiple protocol layers using software-defined reconfigurability in modern wireless infrastructure.
• To design a feedback loop that maps network configuration choices to application-level Quality of Experience (QoE) and uses RL to optimize this mapping.
• To demonstrate that RL-driven scheduling outperforms static and heuristic-based policies in high-load wireless environments for video streaming.

Proposed method

• QFlow models the network scheduling problem as a Markov Decision Process (MDP) with unknown parameters, where actions are queue assignment decisions for clients.
• It implements both model-free RL (e.g., deep Q-learning) and model-based RL (e.g., predictive policy optimization) to learn optimal scheduling policies from QoE feedback.
• The system measures QoE in real time using standard video quality metrics (e.g., MOS) and uses this feedback to train RL agents to improve future decisions.
• Reconfigurable queueing mechanisms at the network edge allow dynamic per-packet and per-flow prioritization based on RL policy outputs.
• The RL agent observes client states (e.g., buffer levels, video quality, channel conditions) and selects queue assignments to maximize cumulative QoE.
• The framework supports both online learning and policy adaptation in real time, enabling responsiveness to changing network loads and user demands.

Experimental results

Research questions

• RQ1 Can reinforcement learning effectively optimize video streaming QoE in wireless networks with dynamic, heterogeneous client demands?
• RQ2 How does RL-based scheduling compare to static or heuristic-based queue assignment policies in high-load scenarios?
• RQ3 To what extent can RL learn to maintain high QoE (e.g., score ≥5) across diverse clients under resource constraints?
• RQ4 Can model-free and model-based RL approaches both achieve stable and scalable performance in real-world wireless environments?
• RQ5 What is the impact of feedback latency and observation granularity on the effectiveness of RL-driven scheduling?

Key findings

• QFlow improves QoE by over 25% compared to the best-known baseline solutions in high-load wireless scenarios.
• The system achieves a perfect QoE score of 5 out of 5 over 85% of the time, significantly outperforming existing approaches.
• Both model-free and model-based RL variants demonstrate stable convergence and effective adaptation to dynamic network conditions.
• The RL-based scheduling policy maintains high performance even under unpredictable client behavior and fluctuating channel conditions.
• The feedback loop between configuration, QoE measurement, and policy update enables continuous optimization without prior knowledge of network dynamics.
• Experimental validation confirms that QFlow’s adaptive queue assignment leads to better fairness and higher overall user satisfaction in real-world wireless deployments.