[Paper Review] DADA: A Large-scale Benchmark and Model for Driver Attention Prediction in Accidental Scenarios.

Jianwu Fang, Dingxin Yan+2 more|arXiv (Cornell University)|Dec 18, 2019

Visual Attention and Saliency DetectionComputer Science54 references19 citations

TL;DR

This paper introduces DADA-2000, a large-scale benchmark with 2000 video sequences for driver attention prediction in normal, critical, and accidental traffic scenarios. It proposes MSAFNet, a multi-path semantic-guided attentive fusion network that models spatio-temporal semantics and scene variations, achieving state-of-the-art performance on the new benchmark with detailed annotations of fixation, saccade, focusing time, accident objects, and categories.

ABSTRACT

Driver attention prediction has recently absorbed increasing attention in traffic scene understanding and is prone to be an essential problem in vision-centered and human-like driving systems. This work, different from other attempts, makes an attempt to predict the driver attention in accidental scenarios containing normal, critical and accidental situations simultaneously. However, challenges tread on the heels of that because of the dynamic traffic scene, intricate and imbalanced accident categories. With the hypothesis that driver attention can provide a selective role of crash-object for assisting driving accident detection or prediction, this paper designs a multi-path semantic-guided attentive fusion network (MSAFNet) that learns the spatio-temporal semantic and scene variation in prediction. For fulfilling this, a large-scale benchmark with 2000 video sequences (named as DADA-2000) is contributed with laborious annotation for driver attention (fixation, saccade, focusing time), accident objects/intervals, as well as the accident categories, and superior performance to state-of-the-arts are provided by thorough evaluations. As far as we know, this is the first comprehensive and quantitative study for the human-eye sensing exploration in accidental scenarios. DADA-2000 is available at this https URL.

Motivation & Objective

• To address the lack of comprehensive, large-scale datasets for driver attention prediction in diverse traffic scenarios, including normal, critical, and accidental conditions.
• To develop a deep learning model capable of capturing complex spatio-temporal semantics and scene variations in dynamic traffic environments.
• To enable quantitative analysis of human-eye sensing behavior in accident-related contexts through detailed, multi-level annotations.
• To establish a benchmark that supports the development and evaluation of vision-centered, human-like driving systems.

Proposed method

• Designing a multi-path semantic-guided attentive fusion network (MSAFNet) to integrate multi-scale spatio-temporal features from video inputs.
• Employing attention mechanisms to dynamically emphasize relevant visual cues and scene variations critical for accident prediction.
• Utilizing a large-scale video dataset (DADA-2000) with human-annotated driver attention (fixation, saccade, focusing time), accident objects, and accident categories.
• Training the MSAFNet model end-to-end on DADA-2000 to predict driver attention under varying traffic conditions.
• Incorporating scene variation modeling to improve robustness across diverse and complex traffic scenarios.
• Validating the model through comprehensive evaluations against state-of-the-art methods on the new benchmark.

Experimental results

Research questions

• RQ1 How can driver attention be effectively predicted across normal, critical, and accidental traffic scenarios in a unified framework?
• RQ2 What role does spatio-temporal semantic understanding play in improving attention prediction accuracy in dynamic traffic scenes?
• RQ3 To what extent can a multi-path attentive fusion network outperform existing models in capturing scene variations and accident-relevant cues?
• RQ4 How does the inclusion of detailed, multi-level annotations (fixation, saccade, focusing time, accident objects) enhance model performance and interpretability?
• RQ5 Can a large-scale, diverse benchmark like DADA-2000 enable more reliable and generalizable evaluation of driver attention prediction models?

Key findings

• The proposed MSAFNet model achieves superior performance compared to state-of-the-art methods on the DADA-2000 benchmark, demonstrating improved accuracy in predicting driver attention across diverse traffic scenarios.
• The integration of multi-path feature extraction and semantic-guided attention mechanisms significantly enhances the model’s ability to detect accident-relevant visual cues.
• The DADA-2000 benchmark provides a comprehensive, large-scale resource with detailed annotations of fixation, saccade, focusing time, accident objects, and categories, enabling more nuanced analysis.
• The model shows robust performance across imbalanced accident categories, indicating effective handling of class imbalance through attentive feature learning.
• The study establishes the first comprehensive and quantitative exploration of human-eye sensing in accidental scenarios, setting a new standard for future research.