[Paper Review] A Hybrid Method for Traffic Flow Forecasting Using Multimodal Deep Learning

Shengdong Du, Tianrui Li+2 more|arXiv (Cornell University)|Mar 6, 2018

Traffic Prediction and Management Techniques31 references67 citations

TL;DR

A hybrid multimodal deep learning framework using 1D CNNs, GRUs, and attention to jointly forecast short-term traffic flow by learning spatial-temporal correlations across multiple data modalities.

ABSTRACT

Traffic flow forecasting has been regarded as a key problem of intelligent transport systems. In this work, we propose a hybrid multimodal deep learning method for short-term traffic flow forecasting, which can jointly and adaptively learn the spatial-temporal correlation features and long temporal interdependence of multi-modality traffic data by an attention auxiliary multimodal deep learning architecture. According to the highly nonlinear characteristics of multi-modality traffic data, the base module of our method consists of one-dimensional Convolutional Neural Networks (1D CNN) and Gated Recurrent Units (GRU) with the attention mechanism. The former is to capture the local trend features and the latter is to capture the long temporal dependencies. Then, we design a hybrid multimodal deep learning framework (HMDLF) for fusing share representation features of different modality traffic data by multiple CNN-GRU-Attention modules. The experimental results indicate that the proposed multimodal deep learning model is capable of dealing with complex nonlinear urban traffic flow forecasting with satisfying accuracy and effectiveness.

Motivation & Objective

• Motivate accurate short-term traffic flow forecasting for intelligent transport systems.
• Address nonlinear characteristics and multi-modality in urban traffic data.
• Learn local trend features and long temporal dependencies within and across modalities.
• Develop a framework that adaptively fuses shared representations from different modalities.

Proposed method

• Base modules combine 1D CNNs for local trend extraction with GRUs for long-term temporal modeling.
• Incorporate attention mechanisms to focus on informative temporal features.
• Design a hybrid multimodal deep learning framework (HMDLF) to fuse shared representations from multiple modality data.
• Utilize multiple CNN-GRU-Attention modules to capture diverse modality-specific patterns.
• Provide an adaptive learning architecture for multimodal fusion across modalities.

Experimental results

Research questions

• RQ1 Can a multimodal deep learning approach improve short-term traffic flow forecasting over single-modality models?
• RQ2 How effectively does attention-based fusion leverage cross-modality information for traffic prediction?
• RQ3 What is the impact of combining CNN and GRU components on capturing local and long-range temporal patterns in traffic data?
• RQ4 Does the proposed HMDLF framework robustly handle nonlinear urban traffic characteristics?

Key findings

• The proposed multimodal deep learning model demonstrates effective handling of complex nonlinear urban traffic flow forecasting.
• The architecture integrates local trend and long-term temporal dependencies through CNN-GRU structures with attention.
• Fusion of multiple modalities via the HMDLF framework yields satisfying accuracy and effectiveness in forecasting.