[Paper Review] DeepStream: Bandwidth Efficient Multi-Camera Video Streaming for Deep Learning Analytics

Hongpeng Guo, Beitong Tian+6 more|arXiv (Cornell University)|Jun 27, 2023

Image and Video Quality Assessment4 citations

TL;DR

DeepStream is a bandwidth-efficient multi-camera video streaming system that reduces transmission overhead via real-time Regions of Interest Detection (ROIDet) on resource-constrained devices and a content-aware bandwidth allocation framework. It achieves up to 54% bandwidth savings with less than 1% accuracy drop and improves detection utility by 23% compared to baselines under constrained bandwidth.

ABSTRACT

Deep learning video analytic systems process live video feeds from multiple cameras with computer vision models deployed on edge or cloud. To optimize utility for these systems, which usually corresponds to query accuracy, efficient bandwidth management for the cameras competing for the fluctuating network resources is crucial. We propose DeepStream, a bandwidth efficient multi-camera video streaming system for deep learning video analytics. DeepStream addresses the challenge of limited and fluctuating bandwidth resources by offering several tailored solutions. We design a novel Regions of Interest detection (ROIDet) algorithm which can run in real time on resource constraint devices, such as Raspberry Pis, to remove spatial redundancy in video frames and reduce the amount of data to be transmitted. We also propose a content-aware bandwidth optimization framework and an Elastic Transmission Mechanism that exploits correlations among video contents. We implement DeepStream on Raspberry Pis and a desktop computer. Evaluations on real-world datasets show that DeepStream's ROIDet algorithm saves up to 54\% bandwidth with less than 1\% accuracy drop. Additionally,DeepStream improves utility by up to 23\% compared to baselines under the same bandwidth conditions.

Motivation & Objective

• Address the challenge of limited and fluctuating bandwidth in multi-camera deep learning video analytics systems.
• Reduce spatial redundancy in video frames without relying on server feedback or heavy models on edge devices.
• Optimize detection utility—defined as object detection accuracy—under constrained bandwidth by leveraging content correlations across cameras.
• Design a system that dynamically allocates bandwidth based on video content and real-time network conditions.
• Enable efficient, real-time processing on low-power devices like Raspberry Pi while maintaining high analytics accuracy.

Proposed method

• Proposes ROIDet, a lightweight, real-time Regions of Interest detection algorithm that runs on resource-constrained devices such as Raspberry Pi without requiring server feedback.
• Introduces a content-aware dynamic programming-based bandwidth allocation algorithm that models detection utility as a function of video content and network conditions.
• Designs an Elastic Transmission Mechanism that exploits spatial and temporal correlations among video streams to adaptively allocate bandwidth.
• Employs offline profiling to establish the relationship between video content characteristics and detection utility for optimization.
• Integrates camera-side processing (ROIDet, compression, feature extraction) with server-side analytics and bandwidth decision-making.
• Uses CRF-based video compression (e.g., CRF=18) to evaluate bandwidth and accuracy trade-offs in end-to-end testing.

Experimental results

Research questions

• RQ1 How can spatial redundancy in multi-camera video streams be effectively reduced on low-power edge devices without relying on server feedback?
• RQ2 To what extent can content-aware bandwidth allocation improve detection utility in multi-camera video analytics under fluctuating network conditions?
• RQ3 Can a lightweight, real-time ROIDet algorithm achieve significant bandwidth savings with minimal impact on detection accuracy?
• RQ4 How do spatial and temporal correlations among video streams influence the effectiveness of elastic bandwidth allocation?
• RQ5 What is the end-to-end latency and performance trade-off of integrating ROIDet and dynamic bandwidth allocation in a real-world deployment?

Key findings

• ROIDet reduces video bandwidth by up to 54% while inducing less than 1% drop in object detection accuracy when compared to full-frame transmission.
• Under identical bandwidth conditions, DeepStream improves detection utility by up to 23% compared to baseline systems that do not use content-aware optimization.
• The system achieves real-time performance on Raspberry Pi devices, with camera-side processing (including ROIDet and compression) completing within a 1-second time slot.
• End-to-end latency is dominated by camera-side processing and transmission, with server-side processing time being a minor contributor.
• CRF-based compression shows that ROIDet-cropped videos achieve ~50% smaller file sizes than original frames while maintaining near-identical detection accuracy.
• The bandwidth allocation framework effectively leverages spatial and temporal correlations across cameras to enhance detection performance under constrained resources.