[Paper Review] FastVA: Deep Learning Video Analytics Through Edge Processing and NPU in Mobile

Tianxiang Tan, Guohong Cao|arXiv (Cornell University)|Jan 13, 2020

IoT and Edge/Fog Computing27 references4 citations

TL;DR

FastVA is a mobile video analytics framework that dynamically combines edge offloading and Neural Processing Unit (NPU) acceleration to optimize deep learning inference. By formulating accuracy- and utility-driven scheduling as integer programs and applying heuristic solutions, FastVA achieves up to 30% higher accuracy than offloading under high network delay and outperforms local execution and baseline methods in both accuracy and frame rate across varying network conditions.

ABSTRACT

Many mobile applications have been developed to apply deep learning for video analytics. Although these advanced deep learning models can provide us with better results, they also suffer from the high computational overhead which means longer delay and more energy consumption when running on mobile devices.To address this issue, we propose a framework called FastVA, which supports deep learning video analytics through edge processing and Neural Processing Unit (NPU) in mobile. The major challenge is to determine when to offload the computation and when to use NPU. Based on the processing time and accuracy requirement of the mobile application, we study two problems: Max-Accuracy where the goal is to maximize the accuracy under some time constraints, and Max-Utility where the goal is to maximize the utility which is a weighted function of processing time and accuracy. We formulate them as integer programming problems and propose heuristics based solutions. We have implemented FastVA on smartphones and demonstrated its effectiveness through extensive evaluations.

Motivation & Objective

• To address the high computational overhead of deep learning models on mobile devices, which leads to long delays and high energy consumption.
• To study the trade-offs between NPU-based local inference (fast but less accurate) and edge offloading (accurate but delay-prone under poor networks).
• To design a framework that intelligently schedules video frames between NPU and edge servers based on real-time network conditions and application-specific accuracy/time requirements.
• To formulate and solve two optimization problems: Max-Accuracy (maximize accuracy under time constraints) and Max-Utility (maximize a weighted function of accuracy and processing time).

Proposed method

• Formulates the Max-Accuracy problem as an integer programming problem to select optimal processing nodes (NPU or edge) per frame under time constraints.
• Proposes a heuristic-based solution for Max-Accuracy that prioritizes offloading when network delay is low and NPU when delay is high, based on frame processing time and accuracy trade-offs.
• Models the Max-Utility problem as a weighted utility function combining accuracy and frame rate, with a tunable parameter α to balance the two.
• Develops an approximation-based heuristic for Max-Utility that adaptively selects between local NPU processing and edge offloading based on network bandwidth and upload delay.
• Uses real-time network monitoring (bandwidth, upload delay) and model characteristics (size, accuracy, execution time on NPU/CPU) to guide scheduling decisions.
• Implements FastVA on smartphones and evaluates it against baseline methods (Local, Offload, DeepDecision, Optimal) under varying frame rates, upload delays, and network conditions.

Experimental results

Research questions

• RQ1 How can deep learning video analytics be accelerated on mobile devices with limited computational resources?
• RQ2 What is the optimal trade-off between NPU-based local inference and edge offloading under varying network conditions?
• RQ3 How can a framework dynamically schedule video frames between NPU and edge servers to maximize accuracy under time constraints?
• RQ4 How can a unified utility metric balancing accuracy and processing time be optimized in real-time mobile video analytics?
• RQ5 What is the performance gain of a hybrid NPU-edge strategy compared to pure local or offloading-only approaches under realistic network variability?

Key findings

• Under high upload delay (e.g., 300 ms), the Offload method suffers a significant accuracy drop (up to 30%) due to frame loss, while FastVA’s Max-Accuracy heuristic maintains high accuracy by switching to NPU processing.
• When network bandwidth is low, the Offload method degrades significantly and can even underperform local processing; FastVA’s Max-Utility heuristic maintains higher utility by adapting to network conditions.
• At 30 fps and 100 ms upload delay, FastVA’s Max-Utility outperforms Offload, Local, and DeepDecision by up to 25% in utility, especially when the trade-off parameter α is high (favoring accuracy).
• As frame rate increases from 30 fps to 50 fps, FastVA maintains high utility and frame rate, outperforming Offload and DeepDecision, particularly when α is low (favoring speed).
• With increasing upload delay, FastVA’s Max-Utility and Max-Accuracy heuristics degrade more slowly than Offload and DeepDecision, demonstrating robustness to network jitter.
• The Optimal solution (computed via exhaustive search) serves as an upper bound, and FastVA’s heuristics achieve 90–95% of this optimal performance in practice.