[Paper Review] Exploring Dynamic Context for Multi-path Trajectory Prediction

Hao Cheng, Wentong Liao+4 more|arXiv (Cornell University)|Oct 30, 2020

Autonomous Vehicle Technology and Safety49 references4 citations

TL;DR

This paper proposes DCENet, a novel framework for multi-path trajectory prediction that leverages self-attention to model dynamic spatial interactions among heterogeneous agents and uses a Conditional Variational Autoencoder (CVAE) to generate diverse, context-aware future trajectories. It achieves state-of-the-art performance on Trajnet and inD benchmarks, demonstrating superior generalization and accuracy in complex traffic scenarios.

ABSTRACT

To accurately predict future positions of different agents in traffic scenarios is crucial for safely deploying intelligent autonomous systems in the real-world environment. However, it remains a challenge due to the behavior of a target agent being affected by other agents dynamically and there being more than one socially possible paths the agent could take. In this paper, we propose a novel framework, named Dynamic Context Encoder Network (DCENet). In our framework, first, the spatial context between agents is explored by using self-attention architectures. Then, the two-stream encoders are trained to learn temporal context between steps by taking the respective observed trajectories and the extracted dynamic spatial context as input. The spatial-temporal context is encoded into a latent space using a Conditional Variational Auto-Encoder (CVAE) module. Finally, a set of future trajectories for each agent is predicted conditioned on the learned spatial-temporal context by sampling from the latent space, repeatedly. DCENet is evaluated on one of the most popular challenging benchmarks for trajectory forecasting Trajnet and reports a new state-of-the-art performance. It also demonstrates superior performance evaluated on the benchmark inD for mixed traffic at intersections. A series of ablation studies is conducted to validate the effectiveness of each proposed module. Our code is available at https://github.com/wtliao/DCENet.

Motivation & Objective

• To improve multi-path trajectory prediction by modeling dynamic, evolving interactions between heterogeneous agents in real-world traffic scenarios.
• To address the limitations of deterministic predictions by generating a diverse set of socially plausible future trajectories.
• To enhance modeling of spatial-temporal dependencies through a two-stream encoder and CVAE-based latent space representation.
• To validate the framework's effectiveness and generalization across diverse traffic environments, including mixed-traffic intersections and shared urban spaces.

Proposed method

• Utilizes self-attention mechanisms to dynamically capture spatial interactions among agents by encoding position, orientation, and speed information into dynamic maps over time.
• Employs two identical two-stream encoders to process observed trajectories and dynamic spatial context separately, learning temporal dependencies across time steps.
• Applies global average pooling to the output of the temporal encoders to summarize the sequence into a compact latent representation.
• Encodes the combined spatial-temporal context into a latent space using a Conditional Variational Autoencoder (CVAE), conditioned on the agent's observed trajectory.
• Generates multiple future trajectories by sampling from the learned latent space, enabling diverse and contextually plausible predictions.
• Trains the entire framework end-to-end using reconstruction and KL divergence losses to optimize trajectory generation quality.

Experimental results

Research questions

• RQ1 How can dynamic spatial interactions between heterogeneous agents be effectively modeled to improve multi-path trajectory prediction?
• RQ2 To what extent does integrating self-attention into trajectory prediction improve performance compared to traditional RNN or LSTM-based methods?
• RQ3 Can a CVAE-based generation module conditioned on learned spatial-temporal context produce diverse, high-quality future trajectory hypotheses?
• RQ4 How does DCENet generalize across diverse real-world traffic scenarios, including complex intersections and shared urban spaces?
• RQ5 What is the contribution of each component (self-attention, two-stream encoders, CVAE) to the overall prediction performance?

Key findings

• DCENet achieves state-of-the-art performance on the Trajnet benchmark, outperforming previous methods in both ADE and FDE metrics for both top-1 and top-10 trajectory predictions.
• On the inD benchmark, DCENet achieves the best performance among compared models in terms of ADE and FDE for the top-10 predictions across all four intersections.
• The model outperforms S-LSTM, S-GAN, and AMENet in most evaluation scenarios, particularly excelling in predicting complex interactions at intersections.
• Ablation studies confirm that the self-attention module and the two-stream encoder design significantly improve performance, with the CVAE module enabling effective generation of diverse trajectories.
• The model demonstrates strong generalization, maintaining high performance on unseen scenarios such as mixed-traffic intersections and pedestrian-heavy urban areas.
• Qualitative results show that predicted trajectory sets accurately reflect agent intent and uncertainty, with the most-likely path closely aligning with ground truth in complex scenarios like roundabouts and zebra crossings.