[Paper Review] Trajectory Forecasting on Temporal Graphs

Görkay Aydemir, Adil Kaan Akan+1 more|arXiv (Cornell University)|Jul 1, 2022

Autonomous Vehicle Technology and Safety4 citations

TL;DR

This paper proposes a temporal graph neural network (TGN) that models dynamic interactions among agents and scene elements over time, incorporating agent-specific and scene-wide memory modules for improved motion forecasting. By learning temporally evolving graph representations and using goal-conditioned trajectory prediction, the method achieves state-of-the-art performance on the Argoverse benchmark, ranking second in b-minFDE and top-3 in minADE and minFDE on both validation and test sets.

ABSTRACT

Predicting future locations of agents in the scene is an important problem in self-driving. In recent years, there has been a significant progress in representing the scene and the agents in it. The interactions of agents with the scene and with each other are typically modeled with a Graph Neural Network. However, the graph structure is mostly static and fails to represent the temporal changes in highly dynamic scenes. In this work, we propose a temporal graph representation to better capture the dynamics in traffic scenes. We complement our representation with two types of memory modules; one focusing on the agent of interest and the other on the entire scene. This allows us to learn temporally-aware representations that can achieve good results even with simple regression of multiple futures. When combined with goal-conditioned prediction, we show better results that can reach the state-of-the-art performance on the Argoverse benchmark.

Motivation & Objective

• To address the limitation of static graph representations in capturing temporal dynamics of traffic scenes.
• To improve motion forecasting by modeling evolving interactions between agents and scene elements over time.
• To enhance prediction accuracy and diversity through a dual-memory mechanism focusing on both the agent of interest and the entire scene.
• To achieve state-of-the-art performance on the Argoverse benchmark without metric-specific optimization.

Proposed method

• Constructs a dynamic temporal graph where each node represents an agent or scene element, and edges are updated based on temporal interactions.
• Uses a graph neural network to propagate features across nodes while explicitly modeling temporal evolution along the time axis.
• Introduces an agent-specific memory module to track the history and motion patterns of the target agent.
• Incorporates a scene-wide memory module to encode global scene dynamics and long-range dependencies.
• Generates multiple future trajectory hypotheses using goal-conditioned prediction, refining predictions based on scene memory and motion context.
• Employs a simple multi-hypothesis regression loss, applying supervision only on the closest predicted trajectory during training.

Experimental results

Research questions

• RQ1 Can a temporal graph representation that evolves over time improve motion forecasting in dynamic traffic scenes?
• RQ2 How does modeling scene-wide temporal dynamics affect prediction accuracy and diversity compared to agent-centric temporal modeling?
• RQ3 To what extent do dual memory modules (agent-specific and scene-wide) improve trajectory forecasting performance?
• RQ4 Can a goal-conditioned prediction head enhance mode coverage without sacrificing endpoint accuracy?

Key findings

• The proposed method achieves a b-minFDE of 1.08 on the Argoverse test set, ranking second among all methods.
• It attains a minFDE of 1.08 and minADE of 0.73 on the validation set, placing it in the top 3 for both metrics.
• The method achieves an MR of 0.10 on the validation set, ranking third, despite not being optimized for this metric.
• Qualitative results show that the temporal graph representation leads to more map-adherent trajectories and better velocity tracking, especially at intersections.
• Goal conditioning significantly improves mode coverage, enabling prediction of left turns and right turns simultaneously, which simple regression misses.
• Ablation studies confirm that both temporal graph learning and dual memory modules are essential for performance gains.