[Paper Review] Collaborative Motion Prediction via Neural Motion Message Passing

Yue Hu, Siheng Chen+2 more|arXiv (Cornell University)|Mar 14, 2020

Autonomous Vehicle Technology and Safety42 references4 citations

TL;DR

This paper proposes Neural Motion Message Passing (NMMP), a graph-based method that learns joint representations for both traffic actors and their directed interactions from observed trajectories. By modeling interactions explicitly and enabling interpretability, NMMP improves motion prediction in pedestrian and joint pedestrian-vehicle scenarios, outperforming prior state-of-the-art methods on multiple benchmarks.

ABSTRACT

Motion prediction is essential and challenging for autonomous vehicles and social robots. One challenge of motion prediction is to model the interaction among traffic actors, which could cooperate with each other to avoid collisions or form groups. To address this challenge, we propose neural motion message passing (NMMP) to explicitly model the interaction and learn representations for directed interactions between actors. Based on the proposed NMMP, we design the motion prediction systems for two settings: the pedestrian setting and the joint pedestrian and vehicle setting. Both systems share a common pattern: we use an individual branch to model the behavior of a single actor and an interactive branch to model the interaction between actors, while with different wrappers to handle the varied input formats and characteristics. The experimental results show that both systems outperform the previous state-of-the-art methods on several existing benchmarks. Besides, we provide interpretability for interaction learning.

Motivation & Objective

• Address the challenge of modeling complex, directed interactions among traffic actors in motion prediction for autonomous systems.
• Improve motion prediction accuracy in both pedestrian-only and joint pedestrian-vehicle scenarios by explicitly learning interaction representations.
• Enable interpretability in interaction learning through disentangled, semantically meaningful interaction embeddings.
• Design a unified framework that generalizes across different motion prediction settings using shared interaction modeling.
• Overcome limitations of prior methods that treat interactions implicitly or fail to model directionality and dynamic interaction patterns.

Proposed method

• Propose Neural Motion Message Passing (NMMP), a message-passing framework adapted to motion prediction that learns joint representations for actors and their pairwise interactions from trajectories.
• Construct an interaction graph where nodes are actors and edges represent directed interactions, with learnable embeddings for both nodes and edges.
• Use a dual-branch architecture: an individual branch models single-actor dynamics, and an interactive branch models inter-actor interactions via NMMP.
• Integrate task-specific wrappers: a GAN-based decoder for open-area pedestrian scenes and a CNN + coordinate transformation for urban driving scenarios with maps and vehicle headings.
• Apply t-SNE to visualize interaction embeddings and correlate them with spatial interaction patterns, enabling interpretability.
• Train end-to-end with supervised loss on future trajectory prediction, using ADE and FDE as evaluation metrics.

Experimental results

Research questions

• RQ1 Can explicit modeling of directed, dynamic interactions between traffic actors improve motion prediction performance in complex scenes?
• RQ2 How does the proposed NMMP module compare to existing graph-based and attention-based interaction modeling methods in terms of accuracy and generalization?
• RQ3 To what extent does the NMMP module enhance performance in high-density crowd scenarios with abundant social interactions?
• RQ4 Can interaction embeddings learned by NMMP be meaningfully interpreted and visualized to reflect real-world interaction patterns?
• RQ5 Does the dual-branch architecture with shared NMMP and task-specific heads generalize across diverse motion prediction settings like pedestrian-only and joint vehicle-pedestrian scenarios?

Key findings

• The proposed PMP-NMMP system outperforms the SOTA SGAN baseline on the ETH-UCY dataset, achieving a 25.4% relative improvement in ADE and 12.7% in FDE on the HOTEL set.
• The double-decoder structure in NMMP significantly outperforms the single-decoder structure, with ADE/FDE of 0.52/1.11 versus 0.56/1.19, indicating better modeling of multi-agent influence.
• The final output of the system surpasses the individual branch output, demonstrating the critical role of the interactive branch in trajectory prediction.
• In crowd scenarios with increasing pedestrian counts, the performance gap between NMMP and SGAN widens, with normalization ratios exceeding 30% for ADE and 25% for FDE in high-density scenes.
• Visualization of interaction embeddings shows that similar interaction patterns (e.g., parallel walking, facing each other) cluster together in the embedding space, confirming interpretability and semantic consistency.
• The NMMP module enables consistent performance gains across both pedestrian and joint pedestrian-vehicle settings, validating its generalization capacity.