[Paper Review] I-SAFE: Instant Suspicious Activity identiFication at the Edge using Fuzzy Decision Making

Seyed Yahya Nikouei, Yu Chen+3 more|arXiv (Cornell University)|Sep 12, 2019

Anomaly Detection Techniques and Applications8 citations

TL;DR

I-SAFE proposes a real-time, edge-based suspicious activity detection system using fuzzy decision making to identify loitering and abnormal behaviors in surveillance video. By fusing lightweight deep learning features with domain-expert-derived fuzzy rules on edge devices, it achieves instant alerts in 0.002 seconds with low latency and minimal network overhead, outperforming state-of-the-art methods in speed and scalability.

ABSTRACT

Urban imagery usually serves as forensic analysis and by design is available for incident mitigation. As more imagery collected, it is harder to narrow down to certain frames among thousands of video clips to a specific incident. A real-time, proactive surveillance system is desirable, which could instantly detect dubious personnel, identify suspicious activities, or raise momentous alerts. The recent proliferation of the edge computing paradigm allows more data-intensive tasks to be accomplished by smart edge devices with lightweight but powerful algorithms. This paper presents a forensic surveillance strategy by introducing an Instant Suspicious Activity identiFication at the Edge (I-SAFE) using fuzzy decision making. A fuzzy control system is proposed to mimic the decision-making process of a security officer. Decisions are made based on video features extracted by a lightweight Deep Machine Learning (DML) model. Based on the requirements from the first-line law enforcement officers, several features are selected and fuzzified to cope with the state of uncertainty that exists in the officers' decision-making process. Using features in the edge hierarchy minimizes the communication delay such that instant alerting is achieved. Additionally, leveraging the Microservices architecture, the I-SAFE scheme possesses good scalability given the increasing complexities at the network edge. Implemented as an edge-based application and tested using exemplary and various labeled dataset surveillance videos, the I-SAFE scheme raises alerts by identifying the suspicious activity in an average of 0.002 seconds. Compared to four other state-of-the-art methods over two other data sets, the experimental study verified the superiority of the I-SAFE decentralized method.

Motivation & Objective

• Address the challenge of real-time detection of suspicious behaviors in urban surveillance video streams, where traditional forensic analysis is too slow.
• Reduce communication overhead and latency in video surveillance by shifting decision-making to the edge using decentralized processing.
• Develop a system that mimics human security officers’ judgment by incorporating expert knowledge into a fuzzy logic decision engine.
• Enable scalable, low-overhead surveillance by leveraging microservices architecture and edge computing for real-time feature extraction and anomaly scoring.
• Minimize false positives and improve robustness in complex, dynamic environments through noise-resistant fuzzy modeling of contextual video features.

Proposed method

• Deploy a lightweight Deep Machine Learning (DML) model on edge devices for real-time human detection and feature extraction from video streams.
• Extract seven spatio-temporal features (e.g., location, time of day, movement patterns, loitering duration) from video frames to contextualize behavior.
• Fuzzify the extracted features using five membership functions to model uncertainty in human judgment, reflecting real-world ambiguity.
• Implement a fuzzy control system with expert-defined rules derived from campus police officers to assess suspiciousness levels per individual.
• Use a microservices architecture to modularize the system, enabling scalability and adaptability to evolving edge network complexities.
• Apply a threshold-based alerting mechanism (60% suspiciousness score) to trigger instant alarms while balancing false positive rates.

Experimental results

Research questions

• RQ1 Can a decentralized, edge-based system detect suspicious behaviors in real time with minimal latency and communication overhead?
• RQ2 How effective is a fuzzy logic system in modeling human-like decision-making for suspicious activity detection compared to purely data-driven models?
• RQ3 To what extent can domain expert knowledge (e.g., from law enforcement) improve detection accuracy and reduce false positives in behavior recognition?
• RQ4 How does the I-SAFE system perform in comparison to state-of-the-art methods in terms of speed, accuracy, and resource efficiency?
• RQ5 Can the system maintain performance under challenging conditions such as occlusion, low resolution, or high crowd density?

Key findings

• The I-SAFE system achieves an average detection latency of 0.002 seconds per frame, enabling instant alerting in real time.
• The system maintains a low false positive rate, with a 60% suspiciousness threshold yielding 4 false positives in the mall dataset and 1 in the subway dataset.
• Compared to four state-of-the-art methods, I-SAFE demonstrates superior performance in speed and scalability while maintaining competitive detection accuracy.
• The fuzzy logic model effectively captures uncertainty in human behavior, improving robustness in complex and dynamic surveillance environments.
• The decoupling of feature extraction from decision-making allows the same fuzzy model to be reused with future, more accurate video processing techniques.
• The system shows strong performance even in low-resolution or occluded scenarios, though detection drops off at greater distances due to insufficient pixel density.