[Paper Review] Anomaly Detection in Residential Video Surveillance on Edge Devices in IoT Framework

Mayur Rajaram Parate, Kishor M. Bhurchandi+1 more|arXiv (Cornell University)|Jul 10, 2021

Anomaly Detection Techniques and Applications36 references4 citations

TL;DR

This paper proposes a lightweight, CPU-only edge computing framework for real-time anomaly detection in residential video surveillance using spatio-temporal feature encoding and trajectory association with two complementary metrics. It achieves high accuracy (AUC up to 0.936 on UMN dataset) and real-time performance (sustained FPS) on low-power edge devices, outperforming existing methods while ensuring privacy and low latency.

ABSTRACT

Intelligent resident surveillance is one of the most essential smart community services. The increasing demand for security needs surveillance systems to be able to detect anomalies in surveillance scenes. Employing high-capacity computational devices for intelligent surveillance in residential societies is costly and not feasible. Therefore, we propose anomaly detection for intelligent surveillance using CPU-only edge devices. A modular framework to capture object-level inferences and tracking is developed. To cope with partial occlusions, posture deformations, and complex scenes, we employed feature encoding and trajectory association governed by two metrices complementing to each other. The elements of an anomaly detection framework are optimized to run on CPU-only edge devices with sufficient frames per second (FPS). The experimental results indicate the proposed method is feasible and achieves satisfactory results in real-life scenarios.

Motivation & Objective

• To develop a cost-effective, real-time anomaly detection system for residential video surveillance using only CPU-based edge devices.
• To address the challenge of processing high-complexity computer vision tasks on resource-constrained edge hardware without relying on GPU or hardware accelerators.
• To ensure real-time performance, low network latency, and data privacy by processing video analytics locally at the edge.
• To optimize the system for low power consumption and high frame rate (FPS) while maintaining high detection accuracy.
• To integrate the anomaly detection system into an IoT framework with LoRa-based alert transmission for real-world deployment.

Proposed method

• The framework uses a modular pipeline for object detection, feature encoding, and trajectory association to enable real-time processing on CPU-only edge devices.
• Spatio-temporal features are encoded using a lightweight neural network (64x32, 128x64, or 256x128 feature size) to represent person identities across frames.
• Trajectory association is governed by two complementary metrics: cosine distance (max-cos-distance) and non-maximum suppression (nms-overlap) to reduce false positives and handle occlusions.
• The system employs a dual-threshold strategy: a high max-cos-distance threshold (0.9) improves robustness against identity swaps, while low nms-overlap (0.3) suppresses overlapping detections from dense crowds.
• The entire pipeline is optimized for CPU execution, with performance evaluated across different feature encoder sizes and threshold configurations.
• Anomalies are detected by analyzing deviations in trajectory patterns and spatial dynamics, with alerts sent via LoRa to preserve privacy and reduce network load.

Experimental results

Research questions

• RQ1 Can anomaly detection in residential video surveillance be effectively performed on CPU-only edge devices without GPU acceleration?
• RQ2 How do different configurations of feature encoder size, max-cos-distance, and nms-overlap affect accuracy and processing speed?
• RQ3 To what extent can spatio-temporal feature encoding and dual-metric trajectory association improve robustness under partial occlusion and posture deformation?
• RQ4 What is the optimal balance between accuracy, frame rate, and power consumption in a real-time edge-based anomaly detection system?
• RQ5 How does the proposed system compare to existing cloud-based or GPU-accelerated methods in terms of AUC and real-time performance on standard datasets?

Key findings

• The proposed system achieves an AUC of 0.918 on the UCSD Ped-1 dataset and 0.936 on the UMN dataset, outperforming existing methods such as [6], [38], [4], and [39].
• Using a 64x32 feature encoder size achieves near-optimal accuracy (AUC ~0.93) with significantly reduced processing time, making it ideal for real-time deployment.
• Setting the nms-overlap threshold to 0.3 effectively suppresses false positive detections from clustered people, improving detection consistency.
• A high max-cos-distance threshold (0.9) reduces identity swaps, especially in scenes with people wearing similar clothing, enhancing trajectory consistency.
• The system maintains real-time performance with sustained frame rates on CPU-only devices, achieving a balance between accuracy and efficiency.
• Without active cooling, CPU core temperature reaches 68°C under load, but remains stable at 59°C with active cooling, indicating thermal viability for long-term operation.