[Paper Review] EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning

Jiachen Li, Fan Yang+2 more|arXiv (Cornell University)|Mar 31, 2020

Autonomous Vehicle Technology and SafetyEngineering47 references101 citations

TL;DR

EvolveGraph introduces a dynamic latent interaction-graph model for multi-agent trajectory forecasting, enabling explicit relational reasoning, multi-modal predictions, and a double-stage training pipeline to achieve state-of-the-art results across synthetic and real-world datasets.

ABSTRACT

Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. In many applications, effective understanding of the situation and accurate trajectory prediction of interactive agents play a significant role in downstream tasks, such as decision making and planning. In this paper, we propose a generic trajectory forecasting framework (named EvolveGraph) with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents. Considering the uncertainty of future behaviors, the model is designed to provide multi-modal prediction hypotheses. Since the underlying interactions may evolve even with abrupt changes, and different modalities of evolution may lead to different outcomes, we address the necessity of dynamic relational reasoning and adaptively evolving the interaction graphs. We also introduce a double-stage training pipeline which not only improves training efficiency and accelerates convergence, but also enhances model performance. The proposed framework is evaluated on both synthetic physics simulations and multiple real-world benchmark datasets in various areas. The experimental results illustrate that our approach achieves state-of-the-art performance in terms of prediction accuracy.

Motivation & Objective

• Motivate accurate trajectory forecasting in multi-agent systems with heterogeneous agents and evolving interactions.
• Develop a framework that explicitly models latent interaction graphs between agents.
• Capture multi-modality and uncertainty in future trajectories through a stochastic, multi-component prediction setup.
• Enable dynamic evolution of interaction graphs to reflect changing relationships over time.
• Improve training efficiency and convergence via a two-stage training pipeline combining static and dynamic graph learning.

Proposed method

• Construct an observation graph with agent nodes and a context node to encode initial interactions.
• Infer a latent interaction graph as a distribution over edge types using a softmax-based encoding with Gumbel-softmax reparameterization.
• Decode future trajectories via a recurrent decoding process that outputs Gaussian mixture distributions (K components) for each agent.
• Introduce a dynamic interaction-graph mechanism that re-encodes the latent graph every tau steps with a graph-recurrent unit to capture evolving relations.
• Use a two-stage training procedure: first train static interaction graphs (stage 1), then fine-tune with dynamic graph evolution (stage 2).
• Adopt a loss design that separately optimizes static and dynamic stages, promoting multi-modality and robustness against mode collapse.

Experimental results

Research questions

• RQ1 Can a latent, heterogeneous interaction graph effectively represent multi-agent relations across time?
• RQ2 Does explicitly evolving the interaction graph improve prediction accuracy over static-relations baselines?
• RQ3 To what extent can multi-modality in future trajectories be captured via Gaussian-mixture decodings and evolving graphs?
• RQ4 Does a two-stage training procedure accelerate convergence and improve performance for dynamic relational reasoning?
• RQ5 How does the approach perform on synthetic and real-world benchmarks with varying agent types and contexts?

Key findings

• EvolveGraph achieves state-of-the-art performance on multiple benchmarks, consistently outperforming baselines across datasets.
• Static interaction-graph learning benefits edge-type accuracy and trajectory prediction, with dynamic evolution providing further gains in changing-relations scenarios.
• The dynamic version (with graph evolution) significantly improves minADE20/minFDE20 on Honda 3D (H3D) and NBA datasets, e.g., a 4.0s reduction of 20.0% in minADE20 and 27.1% in minFDE20 over STGAT baselines.
• Double-stage training accelerates convergence and enhances prediction accuracy compared to single-stage or purely static approaches.
• The model supports multi-modal trajectory prediction by using Gaussian mixtures and evidence of multiple plausible future modalities, reflected in predicted diverse trajectories and evolving graphs.