[Paper Review] Few-shot Scene-adaptive Anomaly Detection

Yiwei Lu, Frank Yu+2 more|arXiv (Cornell University)|Jul 15, 2020

Anomaly Detection Techniques and Applications38 references4 citations

TL;DR

This paper proposes a few-shot scene-adaptive anomaly detection method using meta-learning to adapt a pre-trained model to a new, unseen surveillance scene with only a few frames. The approach achieves state-of-the-art performance on multiple benchmark datasets, significantly outperforming fine-tuning baselines and demonstrating strong generalization across diverse scenes and modalities, including depth images.

ABSTRACT

We address the problem of anomaly detection in videos. The goal is to identify unusual behaviours automatically by learning exclusively from normal videos. Most existing approaches are usually data-hungry and have limited generalization abilities. They usually need to be trained on a large number of videos from a target scene to achieve good results in that scene. In this paper, we propose a novel few-shot scene-adaptive anomaly detection problem to address the limitations of previous approaches. Our goal is to learn to detect anomalies in a previously unseen scene with only a few frames. A reliable solution for this new problem will have huge potential in real-world applications since it is expensive to collect a massive amount of data for each target scene. We propose a meta-learning based approach for solving this new problem; extensive experimental results demonstrate the effectiveness of our proposed method.

Motivation & Objective

• To address the limitation of existing anomaly detection models that require large-scale data from each target scene and fail to generalize across different camera locations.
• To propose a novel few-shot scene-adaptive anomaly detection problem where a model is adapted to a new scene using only a few frames, closer to real-world deployment constraints.
• To develop a meta-learning framework that enables fast adaptation to unseen scenes while maintaining high detection accuracy.
• To demonstrate generalization across diverse scenes, including those with different modalities such as depth images.

Proposed method

• The method uses meta-learning to train a model on a distribution of tasks, each simulating few-shot adaptation to a new scene using data from different camera locations.
• During meta-training, the model learns to quickly adapt to a new scene by optimizing for fast adaptation using only a few support frames from that scene.
• The backbone architecture is an r-GAN model trained for frame reconstruction, which is fine-tuned during inference using a small number of frames from the target scene.
• The meta-learning objective is optimized by sampling multiple scenes per training epoch to prevent overfitting to any single scene.
• During inference, the model adapts to a new scene by updating its parameters using only K frames from that scene, then performs anomaly detection on the rest of the video.
• The approach leverages a task-specific adaptation mechanism that updates the model parameters via a few-shot adaptation protocol, minimizing the reconstruction error on normal frames.

Experimental results

Research questions

• RQ1 Can a model be effectively adapted to a new, previously unseen surveillance scene using only a few frames, without requiring extensive retraining?
• RQ2 How does meta-learning improve generalization across diverse camera scenes compared to standard fine-tuning?
• RQ3 To what extent can the model generalize to scenes with different visual characteristics, such as depth images or significantly different environments?
• RQ4 Does increasing the number of sampled scenes per meta-training epoch improve model robustness and performance?

Key findings

• On the Shanghai Tech dataset, the proposed method achieved an AUC of 80.60 with K=1 and 82.38 with K=10, outperforming the fine-tuned baseline by over 2% on average.
• On the more challenging UCF Crime dataset, the method achieved a 20%+ improvement over baselines in cross-dataset testing, demonstrating strong generalization.
• The model successfully adapted to the UR Fall dataset, which uses depth images and has a drastically different scene from training data, indicating robustness to modality shifts.
• Using five sampled tasks per meta-training epoch (N=5) yielded better performance than using only one task (N=1), showing that multi-scene sampling improves generalization.
• Even with only one support frame (K=1), the meta-learned model outperformed standard fine-tuning, proving the effectiveness of the meta-learning adaptation mechanism.