[Paper Review] Smart City Transportation: Deep Learning Ensemble Approach for Traffic Accident Detection

Victor Adewopo, Nelly Elsayed|arXiv (Cornell University)|Oct 16, 2023

Traffic Prediction and Management TechniquesEngineering3 citations

TL;DR

This paper proposes a lightweight I3D-CONVLSTM2D model that fuses RGB video frames and optical flow for real-time traffic accident detection in smart cities. Trained on a novel, curated dataset, the model achieves 87% mean average precision (MAP), outperforming baselines while being optimized for edge deployment on devices like Raspberry Pi.

ABSTRACT

The dynamic and unpredictable nature of road traffic necessitates effective accident detection methods for enhancing safety and streamlining traffic management in smart cities. This paper offers a comprehensive exploration study of prevailing accident detection techniques, shedding light on the nuances of other state-of-the-art methodologies while providing a detailed overview of distinct traffic accident types like rear-end collisions, T-bone collisions, and frontal impact accidents. Our novel approach introduces the I3D-CONVLSTM2D model architecture, a lightweight solution tailored explicitly for accident detection in smart city traffic surveillance systems by integrating RGB frames with optical flow information. Our experimental study's empirical analysis underscores our approach's efficacy, with the I3D-CONVLSTM2D RGB + Optical-Flow (Trainable) model outperforming its counterparts, achieving an impressive 87\% Mean Average Precision (MAP). Our findings further elaborate on the challenges posed by data imbalances, particularly when working with a limited number of datasets, road structures, and traffic scenarios. Ultimately, our research illuminates the path towards a sophisticated vision-based accident detection system primed for real-time integration into edge IoT devices within smart urban infrastructures.

Motivation & Objective

• To develop a real-time, computationally efficient accident detection system suitable for deployment on edge IoT devices in smart city infrastructures.
• To address the challenge of data imbalance and limited diversity in existing accident detection datasets by curating a specialized benchmark dataset from traffic and dashcam footage.
• To improve detection accuracy by integrating spatio-temporal features from both RGB frames and optical flow using a novel deep learning architecture.
• To distinguish between genuine traffic accidents and similar-looking events such as stationary vehicles or slow-moving traffic, enhancing system robustness.
• To provide a lightweight, trainable alternative to heavy models like I3D, enabling practical real-world implementation on resource-constrained hardware.

Proposed method

• The proposed I3D-CONVLSTM2D model combines 3D Convolutional Neural Networks (I3D) with ConvLSTM2D layers to extract spatio-temporal features from video sequences.
• The model processes both RGB frames and optical flow inputs, leveraging feature fusion to enhance detection of dynamic accident-related motions.
• Transfer learning is applied using pre-trained I3D weights on the Kinetics dataset to improve feature representation and reduce training time.
• A two-stream architecture is employed, with separate branches for RGB and optical flow inputs, followed by late fusion before classification.
• The model is trained on a custom dataset of 1,000+ video clips covering diverse accident types (rear-end, T-bone, frontal impact) and road conditions.
• Inference is optimized for edge deployment using model compression techniques, with a total parameter count of 9 million for the RGB + optical flow variant.

Figure 1: Percentage distribution of different types of collision.

Experimental results

Research questions

• RQ1 Can a lightweight deep learning model effectively detect traffic accidents in real time using only RGB and optical flow inputs from surveillance cameras?
• RQ2 How does the fusion of RGB and optical flow improve accident detection accuracy compared to using either modality alone?
• RQ3 To what extent can a custom-curated dataset enhance model generalization and performance on rare accident types?
• RQ4 How does the I3D-CONVLSTM2D architecture outperform existing state-of-the-art models in terms of accuracy and computational efficiency?
• RQ5 What are the primary failure modes of the system, and how can they be mitigated in real-world deployment?

Key findings

• The I3D-CONVLSTM2D RGB + Optical-Flow (Trainable) model achieved a mean average precision (MAP) of 87%, significantly outperforming other baseline models.
• The model demonstrated 80% accuracy in classifying accident events, with a precision, recall, and F1 score of 75% in the Transformer-based variant, though with longer training times.
• The lightweight architecture, with only 9 million parameters, was successfully trained on a 2-GPU setup (11 GB per GPU), enabling efficient edge deployment.
• The study identified that stationary vehicles and slow-moving traffic often trigger false positives, indicating a need for improved contextual understanding.
• Environmental factors such as dust, smoke, and poor visibility were found to degrade model performance, highlighting the need for robustness enhancements.
• The curated dataset of diverse accident scenarios and road types proved valuable for training and generalization, offering a resource for future research.

Figure 2: Accidents by Intersection Type and Light Condition Associated.