[Paper Review] AQuA: Analytical Quality Assessment for Optimizing Video Analytics Systems

Sibendu Paul, Utsav Drolia+2 more|arXiv (Cornell University)|Jan 24, 2021

Image Enhancement Techniques63 references4 citations

TL;DR

AQuA introduces an analytical quality assessment framework that scores video frame distortions to protect deep learning analytics accuracy by filtering low-quality frames early at the edge. Using a lightweight CNN-based feature extractor and a novel 'classifier opinion score' metric, AQuA reduces high-confidence errors by 17% and cuts computation and bandwidth usage by 25% with only 14ms overhead.

ABSTRACT

Millions of cameras at edge are being deployed to power a variety of different deep learning applications. However, the frames captured by these cameras are not always pristine - they can be distorted due to lighting issues, sensor noise, compression etc. Such distortions not only deteriorate visual quality, they impact the accuracy of deep learning applications that process such video streams. In this work, we introduce AQuA, to protect application accuracy against such distorted frames by scoring the level of distortion in the frames. It takes into account the analytical quality of frames, not the visual quality, by learning a novel metric, classifier opinion score, and uses a lightweight, CNN-based, object-independent feature extractor. AQuA accurately scores distortion levels of frames and generalizes to multiple different deep learning applications. When used for filtering poor quality frames at edge, it reduces high-confidence errors for analytics applications by 17%. Through filtering, and due to its low overhead (14ms), AQuA can also reduce computation time and average bandwidth usage by 25%.

Motivation & Objective

• To address the growing problem of video analytics errors caused by natural distortions from camera hardware and transmission.
• To reduce high-confidence misclassifications in deep learning models due to low-quality input frames.
• To minimize redundant computation and bandwidth usage in edge-based video analytics pipelines.
• To enable early detection and filtering of poor-quality frames before they reach downstream analytics models.
• To generalize across multiple video analytics applications without requiring model retraining.

Proposed method

• AQuA uses a truncated Inception-v3 network as a lightweight, object-independent feature extractor to analyze frame-level distortions.
• It introduces a novel 'classifier opinion score' metric that reflects model confidence on distorted inputs, replacing human perceptual quality assessment.
• The framework trains a regression model to predict distortion levels based on features and classifier opinion scores.
• AQuA is deployed at the edge to filter frames before they are processed by analytics models, reducing downstream resource consumption.
• It generalizes across multiple deep learning applications by focusing on distortion impact rather than object-specific features.
• The system is designed for low latency (14ms) and high scalability in real-time video analytics workloads.

Experimental results

Research questions

• RQ1 Can a frame-level quality assessment system reduce high-confidence errors in video analytics applications caused by natural distortions?
• RQ2 How effectively can a lightweight, CNN-based model predict distortion levels without relying on human opinion scores?
• RQ3 To what extent can AQuA generalize across diverse video analytics applications without retraining?
• RQ4 What is the impact of early frame filtering on compute and bandwidth usage in edge video pipelines?
• RQ5 Can AQuA detect and predict camera misconfigurations that lead to quality degradation?

Key findings

• AQuA reduces high-confidence errors in video analytics applications by 17% through early filtering of distorted frames.
• The system reduces average computation time and bandwidth usage by 25% by eliminating processing of low-quality frames.
• AQuA achieves a 14ms inference latency, making it suitable for real-time edge deployment.
• The classifier opinion score metric correlates more strongly with application confidence than traditional IQA metrics.
• AQuA generalizes across multiple video analytics applications without requiring model fine-tuning or retraining.
• The framework outperforms state-of-the-art IQA methods in predicting application-level accuracy degradation.