[Paper Review] When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos

Yu Yao, Xizi Wang+4 more|arXiv (Cornell University)|Apr 6, 2020

Anomaly Detection Techniques and Applications46 references38 citations

TL;DR

Introduces the DoTA dataset for egocentric traffic video anomaly detection with temporal, spatial, and categorical annotations, and proposes the STAUC spatio-temporal evaluation metric to better assess VAD performance.

ABSTRACT

Video anomaly detection (VAD) has been extensively studied. However, research on egocentric traffic videos with dynamic scenes lacks large-scale benchmark datasets as well as effective evaluation metrics. This paper proposes traffic anomaly detection with a extit{when-where-what} pipeline to detect, localize, and recognize anomalous events from egocentric videos. We introduce a new dataset called Detection of Traffic Anomaly (DoTA) containing 4,677 videos with temporal, spatial, and categorical annotations. A new spatial-temporal area under curve (STAUC) evaluation metric is proposed and used with DoTA. State-of-the-art methods are benchmarked for two VAD-related tasks.Experimental results show STAUC is an effective VAD metric. To our knowledge, DoTA is the largest traffic anomaly dataset to-date and is the first supporting traffic anomaly studies across when-where-what perspectives. Our code and dataset can be found in: https://github.com/MoonBlvd/Detection-of-Traffic-Anomaly

Motivation & Objective

• Provide a large-scale ego-centric traffic video dataset supporting the when-where-what analysis of anomalies.
• Introduce rich temporal, spatial, and categorical annotations for driving anomalies.
• Propose the STAUC metric to evaluate spatio-temporal localization of anomalies.
• Benchmark existing VAD and VAR methods on the DoTA dataset to establish baselines.
• Investigate the benefits of ensemble approaches combining frame-level and object-centric cues.

Proposed method

• Dataset construction with 4,677 videos at 1280x720 resolution and 10 fps extraction for annotations.
• Rich annotations including anomaly type (18 categories after ego/non-ego split), temporal start/end, and per-frame anomalous object tracklets.
• Definition of the When-Where-What pipeline and framing VAD as predicting per-frame anomaly scores and per-object anomalous scores.
• Introduction of STAUC, a spatio-temporal extension of AUC that weights true positives by overlap with annotated anomalous regions.
• Benchmarking of unsupervised frame-level, unsupervised object-centric, and supervised VAD methods on DoTA.
• Benchmarking of VAR methods (C3D, I3D, R3D, MC3D, R(2+1)D, TSN, SlowFast) on DoTA.

Experimental results

Research questions

• RQ1 Can a large-scale egocentric driving video dataset support comprehensive when-where-what anomaly analysis?
• RQ2 Is the STAUC metric a more informative evaluator than AUC for VAD in driving videos?
• RQ3 How do state-of-the-art VAD methods perform on DoTA under both AUC and STAUC?
• RQ4 How do VAR methods perform on driving anomalies and what are the challenges?
• RQ5 Does ensemble learning improve VAD performance by combining frame-level appearance and object-centric cues?

Key findings

• DoTA is the largest traffic anomaly dataset to date with temporal, spatial, and categorical annotations for when-where-what analysis.
• STAUC generally outperforms AUC, highlighting better localization of anomalous regions.
• Ensemble methods combining frame-level and object-centric cues achieve the best AUC and STAUC among tested VAD approaches.
• Supervised VAR models achieve higher AUC than unsupervised VAD but overall VAR accuracy on DoTA remains challenging with notable per-class variability.
• Frame-level and object-centric VAD methods can partially compensate each other, with foreground-focused variants improving spatial localization.
• DoTA benchmarks indicate Traffic anomaly understanding remains difficult, motivating further research.