[Paper Review] GraphTCN: Spatio-Temporal Interaction Modeling for Human Trajectory Prediction

Chengxin Wang, Shaofeng Cai+1 more|arXiv (Cornell University)|Mar 16, 2020

Autonomous Vehicle Technology and Safety56 references4 citations

TL;DR

GraphTCN proposes a novel CNN-based spatio-temporal graph network that models human trajectory prediction using an edge feature-based graph attention network (EFGAT) for spatial interactions and a gated temporal convolutional network (TCN) for temporal dynamics. It achieves state-of-the-art accuracy with 19.4% lower average displacement error and 5.22x faster inference than prior methods, while reducing predicted paths by 5x.

ABSTRACT

Predicting the future paths of an agent's neighbors accurately and in a timely manner is central to the autonomous applications for collision avoidance. Conventional approaches, e.g., LSTM-based models, take considerable computational costs in the prediction, especially for the long sequence prediction. To support more efficient and accurate trajectory predictions, we propose a novel CNN-based spatial-temporal graph framework GraphTCN, which models the spatial interactions as social graphs and captures the spatio-temporal interactions with a modified temporal convolutional network. In contrast to conventional models, both the spatial and temporal modeling of our model are computed within each local time window. Therefore, it can be executed in parallel for much higher efficiency, and meanwhile with accuracy comparable to best-performing approaches. Experimental results confirm that our model achieves better performance in terms of both efficiency and accuracy as compared with state-of-the-art models on various trajectory prediction benchmark datasets.

Motivation & Objective

• To address the high computational cost and slow inference of RNN-based trajectory prediction models in real-time autonomous applications.
• To improve modeling of social interactions by incorporating relative spatial dynamics and asymmetric influence between agents.
• To develop a fully feed-forward architecture that enables parallel inference while maintaining high accuracy.
• To reduce prediction error and inference time compared to state-of-the-art methods on benchmark datasets.

Proposed method

• GraphTCN uses a graph structure where nodes represent agents and edges encode relative spatial relations using edge features.
• It introduces EFGAT (Edge Feature-based Graph Attention Network) that learns adaptive attention weights based on relative movement trends and proximity.
• The temporal modeling employs a gated TCN (Temporal Convolutional Network) with highway connections to capture long-range sequence dynamics efficiently.
• The model processes spatial and temporal interactions within local time windows, enabling parallel computation and high inference speed.
• Skip connections and gated mechanisms in EFGAT and TCN help stabilize training and improve gradient flow.
• The framework is end-to-end trainable and designed for real-time deployment on resource-constrained devices.

Experimental results

Research questions

• RQ1 Can a fully feed-forward architecture outperform RNN-based models in trajectory prediction accuracy while achieving significantly faster inference?
• RQ2 How effectively can relative spatial dynamics (e.g., direction, velocity) be modeled as edge features in a graph attention mechanism?
• RQ3 Does asymmetric attention between interacting agents improve prediction quality in complex crowd scenarios?
• RQ4 Can local window-based spatio-temporal processing maintain accuracy while enabling parallel inference?

Key findings

• GraphTCN reduces average displacement error by 19.4% and final displacement error by 13.6% compared to state-of-the-art models on benchmark datasets.
• The model achieves up to 5.22x wall-clock time speedup over existing solutions, enabling real-time deployment on low-resource devices.
• With only 5 predicted paths, GraphTCN outperforms models generating 20 paths, indicating higher sample efficiency.
• Visualizations show that GraphTCN generates more accurate, collision-free, and socially compliant trajectories, especially in complex scenarios like pedestrian merging or opposing directions.
• EFGAT learns asymmetric attention weights that reflect real-world interaction dynamics, such as higher influence from oncoming or faster-moving agents.
• The model successfully predicts both expected and unexpected behaviors, such as sudden direction changes, with diverse and plausible future paths.