[Paper Review] Focus: Querying Large Video Datasets with Low Latency and Low Cost

Kevin Hsieh, Ganesh Ananthanarayanan+5 more|arXiv (Cornell University)|Jan 10, 2018

Advanced Neural Network ApplicationsComputer Science42 references97 citations

TL;DR

Focus enables low-latency, low-cost querying over large video datasets by ingesting with cheap, specialized CNNs, clustering similar objects, and validating with expensive GT-CNN at query time.

ABSTRACT

Large volumes of videos are continuously recorded from cameras deployed for traffic control and surveillance with the goal of answering "after the fact" queries: identify video frames with objects of certain classes (cars, bags) from many days of recorded video. While advancements in convolutional neural networks (CNNs) have enabled answering such queries with high accuracy, they are too expensive and slow. We build Focus, a system for low-latency and low-cost querying on large video datasets. Focus uses cheap ingestion techniques to index the videos by the objects occurring in them. At ingest-time, it uses compression and video-specific specialization of CNNs. Focus handles the lower accuracy of the cheap CNNs by judiciously leveraging expensive CNNs at query-time. To reduce query time latency, we cluster similar objects and hence avoid redundant processing. Using experiments on video streams from traffic, surveillance and news channels, we see that Focus uses 58X fewer GPU cycles than running expensive ingest processors and is 37X faster than processing all the video at query time.

Motivation & Objective

• Motivate the need for inexpensive, fast querying over long video datasets for after-the-fact analyses.
• Propose a cost-aware ingest-and-query framework that preserves high precision/recall relative to a ground-truth CNN (GT-CNN).
• Develop techniques (cheap ingest CNNs, top-K indexing, clustering, and video-specific specialization) to meet target accuracy with reduced cost and latency.
• Provide a tunable system that trades ingest cost against query latency based on application needs.
• Demonstrate that Focus yields significant cost and latency improvements across multiple video domains.

Proposed method

• Use cheap, compressed, and specialized CNNs at ingest to detect and index objects by class.
• Index each object with the top-K results from CheapCNNs to recover recall.
• Cluster similar objects using cheap CNN feature vectors to reduce the number of GT-CNN evaluations at query time.
• Classify cluster centroids with the GT-CNN at query time to assign labels to all objects in the cluster, reducing latency.
• Specialize ingest CNNs to each video stream by retraining reduced models on frequent classes, creating a small, accurate, stream-specific model set.
• Balance ingest cost and query latency by selecting CheapCNN, K, Ls (popular classes), and clustering threshold T to meet user-defined precision/recall targets.

Experimental results

Research questions

• RQ1 How can we achieve low-cost ingest while preserving high accuracy relative to GT-CNN for video queries?
• RQ2 What combination of cheap ingest CNNs, top-K indexing, clustering, and specialization yields best trade-offs between ingestion cost and query latency?
• RQ3 How does video-specific specialization affect recall, precision, and performance across domains (traffic, surveillance, news)?
• RQ4 Can clustering and centroid-based GT-CNN evaluation substantially reduce query-time computation without sacrificing accuracy?
• RQ5 What parameters best balance ingest cost, query latency, and accuracy for different target applications?

Key findings

• Focus achieves up to 58x cheaper ingest compared to Ingest-all and up to 98x in some cases.
• Focus is up to 37x faster than Query-all in query latency.
• Focus maintains at least 95% precision and recall in representative evaluations.
• Ingest cost can drop from $250/month/stream to $4/month/stream with Focus.
• Query latency for a 24-hour video can drop from around 1 hour to under 2 minutes.
• Specialization and top-K indexing enable high recall with much smaller K values (e.g., K as small as 2–4 for specialized models).