[Paper Review] Driving Anomaly Detection Using Conditional Generative Adversarial Network

Yuning Qiu, Teruhisa Misu+1 more|arXiv (Cornell University)|Mar 15, 2022

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes an unsupervised driving anomaly detection method using a conditional generative adversarial network (cGAN) that predicts future CAN-Bus and physiological signals from past observations. The anomaly score is derived from the discriminator's output difference between predicted and actual signals, achieving high detection of hazardous, low-frequency events without predefined rules or labeled anomalies.

ABSTRACT

Anomaly driving detection is an important problem in advanced driver assistance systems (ADAS). It is important to identify potential hazard scenarios as early as possible to avoid potential accidents. This study proposes an unsupervised method to quantify driving anomalies using a conditional generative adversarial network (GAN). The approach predicts upcoming driving scenarios by conditioning the models on the previously observed signals. The system uses the difference of the output from the discriminator between the predicted and actual signals as a metric to quantify the anomaly degree of a driving segment. We take a driver-centric approach, considering physiological signals from the driver and controller area network-Bus (CAN-Bus) signals from the vehicle. The approach is implemented with convolutional neural networks (CNNs) to extract discriminative feature representations, and with long short-term memory (LSTM) cells to capture temporal information. The study is implemented and evaluated with the driving anomaly dataset (DAD), which includes 250 hours of naturalistic recordings manually annotated with driving events. The experimental results reveal that recordings annotated with events that are likely to be anomalous, such as avoiding on-road pedestrians and traffic rule violations, have higher anomaly scores than recordings without any event annotation. The results are validated with perceptual evaluations, where annotators are asked to assess the risk and familiarity of the videos detected with high anomaly scores. The results indicate that the driving segments with higher anomaly scores are more risky and less regularly seen on the road than other driving segments, validating the proposed unsupervised approach.

Motivation & Objective

• To develop an unsupervised anomaly detection system for driving that identifies hazardous deviations before they lead to accidents.
• To overcome limitations of rule-based or threshold-based systems that react only after errors occur.
• To leverage multimodal data—CAN-Bus signals and driver physiological signals (ECG, EDA, BR)—for comprehensive anomaly detection.
• To validate the model’s effectiveness using perceptual evaluations and manual annotations from the Driving Anomaly Dataset (DAD).

Proposed method

• Uses a conditional GAN framework where the generator predicts future signals (CAN-Bus and physiological) conditioned on past observed signals.
• Employs convolutional neural networks (CNNs) to extract spatial and temporal features from raw sensor signals.
• Integrates long short-term memory (LSTM) networks to model sequential dependencies and temporal dynamics in driving sequences.
• Computes anomaly scores as the difference in discriminator output between real and generated future signals, with higher values indicating greater deviation.
• Trains the model end-to-end in an unsupervised manner, using only normal driving data for learning.
• The model is evaluated on the DAD dataset, which includes 250 hours of naturalistic driving with manual annotations of hazardous events.

Experimental results

Research questions

• RQ1 Can a conditional GAN framework effectively detect driving anomalies without relying on predefined rules or labeled anomalies?
• RQ2 How well does the model detect events that are objectively hazardous but not easily captured by traditional rule-based systems?
• RQ3 To what extent do multimodal inputs (CAN-Bus and physiological signals) improve anomaly detection performance compared to single-modality inputs?
• RQ4 How do perceptual evaluations of high-anomaly-score segments compare to random segments in terms of perceived risk and familiarity?

Key findings

• Driving segments annotated with high-risk events—such as pedestrian avoidance and traffic violations—consistently received higher anomaly scores than non-annotated segments.
• The CNN+LSTM-based cGAN model detected 33% of the top 100 high-anomaly segments as 'risky' or 'very risky', compared to only 27.7% for the LSTM-only model and 22.7% for the CNN-only model.
• 33% of the top 100 segments selected by the CNN+LSTM model were rated as occurring 'never' or 'almost never' on roads, indicating strong detection of rare, abnormal events.
• Perceptual evaluations confirmed that high-anomaly-score segments were perceived as significantly riskier (64.7% rated as risky or very risky) and less familiar than randomly selected segments.
• The CNN+LSTM model outperformed both the CNN-only and LSTM-only variants in identifying hazardous and infrequent driving conditions.
• The model’s performance was validated on the DAD dataset, demonstrating that unsupervised anomaly detection is feasible using only normal driving data and multimodal signals.