[Paper Review] Leveraging Future Relationship Reasoning for Vehicle Trajectory Prediction

Dae-Hee Park, Hobin Ryu+4 more|arXiv (Cornell University)|May 24, 2023

Autonomous Vehicle Technology and Safety18 citations

TL;DR

The paper introduces Future Relationship Modeling (FRM) that uses lane-level waypoint occupancy to probabilistically infer interactions between vehicles, enabling diverse multi-modal trajectory predictions and achieving state-of-the-art performance on nuScenes for long-range forecasting.

ABSTRACT

Understanding the interaction between multiple agents is crucial for realistic vehicle trajectory prediction. Existing methods have attempted to infer the interaction from the observed past trajectories of agents using pooling, attention, or graph-based methods, which rely on a deterministic approach. However, these methods can fail under complex road structures, as they cannot predict various interactions that may occur in the future. In this paper, we propose a novel approach that uses lane information to predict a stochastic future relationship among agents. To obtain a coarse future motion of agents, our method first predicts the probability of lane-level waypoint occupancy of vehicles. We then utilize the temporal probability of passing adjacent lanes for each agent pair, assuming that agents passing adjacent lanes will highly interact. We also model the interaction using a probabilistic distribution, which allows for multiple possible future interactions. The distribution is learned from the posterior distribution of interaction obtained from ground truth future trajectories. We validate our method on popular trajectory prediction datasets: nuScenes and Argoverse. The results show that the proposed method brings remarkable performance gain in prediction accuracy, and achieves state-of-the-art performance in long-term prediction benchmark dataset.

Motivation & Objective

• Incorporate road structure to better reason about future interactions between vehicles.
• Model stochastic, multi-modal interactions rather than deterministic relations.
• Leverage lane-level information to predict future relationships and improve trajectory forecasting.
• Train with CVAE to capture uncertainty in interactions and future paths.

Proposed method

• Predict waypoint occupancy along lane segments as a probabilistic representation of future motion.
• Infer Future Relationship via a Gaussian Mixture distribution over inter-vehicle interactions.
• Use Graph Convolutional Networks to smooth waypoint occupancy and compute inter-agent proximity across lane topology.
• Apply a CVAE framework to jointly model intention (goal) and interaction (future relationship) for decoding future trajectories.
• Train with ELBO optimization including a reconstruction term to mitigate edge-degeneracy in neural relational inference.

Experimental results

Research questions

• RQ1 How can lane information be leveraged to predict future interactions between vehicles?
• RQ2 Can a probabilistic, multi-modal model of future relationships improve trajectory prediction over deterministic interaction modeling?
• RQ3 Does explicit modeling of future relationships yield better long-range trajectory predictions on real-world datasets?

Key findings

• The proposed method achieves state-of-the-art performance for long-range prediction on nuScenes.
• Explicit future relationship modeling via waypoint occupancy improves interaction understanding and samples’ plausibility compared to past-trajectory-based interaction methods.
• Multi-modal stochastic interaction (Gaussian Mixture) yields better results than single-modality priors.
• Ablation studies show that future relationship modeling and GCN-based proximity estimation are crucial for performance gains.
• Qualitative results demonstrate diverse, interaction-aware trajectory samples that reflect reasonable future interactions.