[Paper Review] FMA-ETA: Estimating Travel Time Entirely Based on FFN With Attention

Yiwen Sun, Yulu Wang+5 more|arXiv (Cornell University)|Jun 7, 2020

Traffic Prediction and Management Techniques27 references4 citations

TL;DR

FMA-ETA proposes a novel, entirely feed-forward network (FFN)-based framework with a Multi-factor self-attention mechanism for travel time estimation, achieving state-of-the-art accuracy while significantly outperforming RNN-based models like WDR in inference speed. It enables fast, accurate real-time ETA prediction by effectively modeling multi-factor sequence dependencies without recurrence.

ABSTRACT

Estimated time of arrival (ETA) is one of the most important services in intelligent transportation systems and becomes a challenging spatial-temporal (ST) data mining task in recent years. Nowadays, deep learning based methods, specifically recurrent neural networks (RNN) based ones are adapted to model the ST patterns from massive data for ETA and become the state-of-the-art. However, RNN is suffering from slow training and inference speed, as its structure is unfriendly to parallel computing. To solve this problem, we propose a novel, brief and effective framework mainly based on feed-forward network (FFN) for ETA, FFN with Multi-factor self-Attention (FMA-ETA). The novel Multi-factor self-attention mechanism is proposed to deal with different category features and aggregate the information purposefully. Extensive experimental results on the real-world vehicle travel dataset show FMA-ETA is competitive with state-of-the-art methods in terms of the prediction accuracy with significantly better inference speed.

Motivation & Objective

• To address the slow inference speed of RNN-based models in real-time travel time estimation, especially in large-scale ride-hailing platforms.
• To explore whether a feed-forward network (FFN) architecture can effectively capture complex spatial-temporal dependencies in sequential travel data without recurrence.
• To design a novel attention mechanism tailored for multi-factor sequential inputs in ETA, improving upon standard multi-head attention used in NLP.
• To achieve competitive prediction accuracy while drastically improving inference speed compared to state-of-the-art RNN-based methods.
• To validate the framework on a massive real-world dataset of over 500 million trajectories from a major ride-sharing platform.

Proposed method

• Proposes FMA-ETA, a deep learning framework for ETA that relies entirely on feed-forward networks (FFN) instead of recurrent structures like LSTM.
• Introduces a novel Multi-factor self-attention mechanism that separately attends to different types of input features (e.g., time, location, traffic) to better model heterogeneous dependencies.
• Applies learnable positional encodings to preserve sequence order in the FFN architecture, replacing recurrence with parallelizable attention computation.
• Uses multi-scale feed-forward blocks with residual connections to enhance representational capacity while maintaining training stability.
• Employs a multi-task learning objective to jointly predict travel time and related route features, improving generalization and robustness.
• Trains the model end-to-end on a large-scale real-world trajectory dataset from a major ride-hailing platform using standard optimization techniques.

Experimental results

Research questions

• RQ1 Can a feed-forward network architecture alone achieve competitive travel time prediction accuracy compared to state-of-the-art RNN-based models?
• RQ2 Does a custom Multi-factor self-attention mechanism outperform standard multi-head attention in modeling complex, multi-faceted sequential dependencies for ETA?
• RQ3 Can an FFN-based model achieve significantly faster inference speed than RNN-based models without sacrificing prediction accuracy?
• RQ4 How does the proposed framework perform across varying sequence lengths, especially long sequences common in real-world ride-hailing data?
• RQ5 Is the Multi-factor self-attention mechanism generalizable to other sequence learning tasks affected by multiple heterogeneous factors?

Key findings

• FMA-ETA achieves prediction accuracy comparable to the state-of-the-art RNN-based method WDR, with slightly better performance on MAE and MAPE metrics.
• FMA-ETA significantly outperforms WDR in inference speed, especially for longer sequences, with a clear advantage when sequence length exceeds 180 time steps.
• The proposed Multi-factor self-attention mechanism demonstrates superior performance over standard multi-head attention in the context of travel time prediction.
• The model maintains high accuracy across diverse road types and traffic conditions, as validated on a real-world dataset of over 500 million trajectories.
• FMA-ETA enables faster real-time inference, making it more suitable for deployment in production-level ride-hailing and navigation systems.
• The framework is robust to long-range dependencies and scales efficiently with sequence length, unlike RNNs whose inference time increases rapidly.