[Paper Review] Modified Supervised Contrastive Learning for Detecting Anomalous Driving Behaviours.

Shehroz S. Khan, Ziting Shen+3 more|arXiv (Cornell University)|Sep 9, 2021

Anomaly Detection Techniques and Applications25 references4 citations

TL;DR

This paper proposes a modified supervised contrastive learning framework that enhances visual representation learning for detecting both seen and unseen anomalous driving behaviors. By retaining the projection head during inference and reweighting negative pairs in the contrastive loss, the method achieves state-of-the-art performance, with a peak AUC ROC of 0.9738 and AUC PR of 0.9772 using fused depth and infrared modalities from top and front cameras on a 783-minute driving dataset.

ABSTRACT

Detecting distracted driving behaviours is important to reduce millions of deaths and injuries occurring worldwide. Distracted or anomalous driving behaviours are deviations from the 'normal' driving that need to be identified correctly to alert the driver. However, these driving behaviours do not comprise of one specific type of driving style and their distribution can be different during training and testing phases of a classifier. We formulate this problem as a supervised contrastive learning approach to learn a visual representation to detect normal, and seen and unseen anomalous driving behaviours. We made a change to the standard contrastive loss function to adjust the similarity of negative pairs to aid the optimization. Normally, the (self) supervised contrastive framework contains an encoder followed by a projection head, which is omitted during testing phase as the encoding layers are considered to contain general visual representative information. However, we assert that for supervised contrastive learning task, including projection head will be beneficial. We showed our results on a Driver Anomaly Detection dataset that contains 783 minutes of video recordings of normal and anomalous driving behaviours of 31 drivers from various from top and front cameras (both depth and infrared). We also performed an extra step of fine tuning the labels in this dataset. Out of 9 video modalities combinations, our modified contrastive approach improved the ROC AUC on 7 in comparison to the baseline models (from 3.12% to 8.91% for different modalities); the remaining two models also had manual labelling. We performed statistical tests that showed evidence that our modifications perform better than the baseline contrastive models. Finally, the results showed that the fusion of depth and infrared modalities from top and front view achieved the best AUC ROC of 0.9738 and AUC PR of 0.9772.

Motivation & Objective

• Address the challenge of detecting diverse and unseen anomalous driving behaviors that deviate from normal driving patterns.
• Overcome distribution shifts between training and testing data for anomalous behaviors, which limits traditional supervised models.
• Improve visual representation learning for anomaly detection by adapting contrastive learning to supervised settings with dynamic negative pair weighting.
• Demonstrate the benefit of retaining the projection head during inference in supervised contrastive learning, contrary to standard practice.
• Achieve robust performance across multiple video modality combinations, including unseen anomalous behaviors.

Proposed method

• Propose a modified contrastive loss function that adjusts the similarity of negative pairs to improve optimization stability and representation learning.
• Retain the projection head during inference, arguing it contributes meaningful feature discrimination for anomaly detection in supervised contrastive learning.
• Fine-tune labels in the Driver Anomaly Detection dataset to improve annotation consistency and model generalization.
• Train a model with an encoder and projection head on multi-modal inputs (depth, infrared, RGB) from front and top cameras.
• Perform early fusion of depth and infrared modalities from both camera views to enhance discriminative feature learning.
• Use contrastive learning to learn representations that separate normal behaviors from seen and unseen anomalous behaviors.

Experimental results

Research questions

• RQ1 Can a modified supervised contrastive learning approach improve detection of both seen and unseen anomalous driving behaviors compared to standard contrastive learning?
• RQ2 Does retaining the projection head during inference improve performance in anomaly detection for driving behavior recognition?
• RQ3 How does the fusion of depth and infrared modalities from top and front cameras affect detection performance?
• RQ4 To what extent does reweighting negative pairs in the contrastive loss improve model optimization and generalization?
• RQ5 Can fine-tuning of dataset labels lead to measurable improvements in model performance on anomaly detection tasks?

Key findings

• The modified contrastive approach improved ROC AUC by 3.12% to 8.91% across different video modality combinations compared to baseline models.
• The fusion of depth and infrared modalities from top and front views achieved the highest performance with an AUC ROC of 0.9738 and AUC PR of 0.9772.
• Statistical tests confirmed that the proposed modifications significantly outperformed baseline contrastive models across multiple configurations.
• The method achieved improved performance on 7 out of 9 video modality combinations, with the remaining two models also benefiting from manual label refinement.
• The inclusion of the projection head during inference was shown to be beneficial, challenging the conventional practice of discarding it post-training.
• The model demonstrated strong generalization to unseen anomalous behaviors, indicating robustness in real-world deployment scenarios.