[Paper Review] Prediction of Satisfied User Ratio for Compressed Video

Haiqiang Wang, Ioannis Katsavounidis+3 more|arXiv (Cornell University)|Oct 30, 2017

Image and Video Quality Assessment11 references3 citations

TL;DR

This paper proposes a machine learning framework to predict the Satisfied User Ratio (SUR) curve for H.264-compressed video using local VMAF quality scores, global aggregation, masking effects, and support vector regression (SVR). The method achieves low prediction error (MAE = 0.038) and high accuracy in estimating the first JND point, with mean absolute error of 1.273 QP for 720p resolution videos.

ABSTRACT

A large-scale video quality dataset called the VideoSet has been constructed recently to measure human subjective experience of H.264 coded video in terms of the just-noticeable-difference (JND). It measures the first three JND points of 5-second video of resolution 1080p, 720p, 540p and 360p. Based on the VideoSet, we propose a method to predict the satisfied-user-ratio (SUR) curves using a machine learning framework. First, we partition a video clip into local spatial-temporal segments and evaluate the quality of each segment using the VMAF quality index. Then, we aggregate these local VMAF measures to derive a global one. Finally, the masking effect is incorporated and the support vector regression (SVR) is used to predict the SUR curves, from which the JND points can be derived. Experimental results are given to demonstrate the performance of the proposed SUR prediction method.

Motivation & Objective

• To predict the Satisfied User Ratio (SUR) curve for H.264-coded video clips using a machine learning approach.
• To improve JND point estimation by modeling user satisfaction rather than predicting JND directly.
• To incorporate local quality degradation and spatial-temporal masking effects into the prediction framework.
• To leverage the large-scale VideoSet dataset for training and validation of the SUR prediction model.
• To enable accurate derivation of JND points from predicted SUR curves for video quality assessment.

Proposed method

• Partition each video clip into local spatial-temporal segments using fixed window sizes (W×H) for temporal and spatial analysis.
• Compute the VMAF quality index for each segment to assess local quality degradation.
• Aggregate local VMAF scores into a global quality degradation feature vector using cumulative distribution over 20 quality levels.
• Extract masking features using spatial randomness (SR) and temporal randomness (TR) from the reference video, forming a 20-dimensional histogram feature vector.
• Concatenate the 20D quality degradation vector and 20D masking feature vector into a 40D input feature vector.
• Train a support vector regressor (SVR) with radial basis function kernel and ε-insensitive loss to predict the SUR curve from the 40D feature vector.

Experimental results

Research questions

• RQ1 Can a machine learning framework accurately predict the Satisfied User Ratio (SUR) curve for H.264-compressed video?
• RQ2 How does incorporating local quality degradation and masking effects improve SUR prediction performance?
• RQ3 To what extent can the first JND point be accurately derived from the predicted SUR curve?
• RQ4 How does prediction accuracy vary across different video resolutions (1080p, 720p, 540p, 360p)?
• RQ5 What is the impact of fixed spatial-temporal segment parameters on prediction error, especially at lower resolutions?

Key findings

• The proposed SUR prediction method achieves a mean absolute error (MAE) of 0.038 on the SUR curve for 720p video clips.
• The prediction error for the first JND point is 1.273 QP on average for 720p resolution, indicating high accuracy in JND estimation.
• Prediction performance degrades slightly at lower resolutions, with a QP error of 1.605 for 360p, likely due to fixed segment window sizes.
• The predicted SUR curves show strong correlation with ground truth, as evidenced by dots closely aligned along the 45-degree line in the JND prediction vs. ground truth plot.
• The cumulative quality degradation curve feature vector effectively captures perceptual quality changes across QP levels.
• The inclusion of masking features based on spatial and temporal randomness improves the model’s ability to predict user satisfaction under complex visual content.