[Paper Review] Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models

J. L. Vázquez Burguete, Alexander Liniger+3 more|arXiv (Cornell University)|Apr 5, 2022

Autonomous Vehicle Technology and Safety29 citations

TL;DR

The paper proposes an interactive motion prediction-planning module that tightly couples prediction and planning via a game-theoretic MPC and a novel interactive multi-agent policy (IMAP) trained with model-based imitation learning.

ABSTRACT

In most classical Autonomous Vehicle (AV) stacks, the prediction and planning layers are separated, limiting the planner to react to predictions that are not informed by the planned trajectory of the AV. This work presents a module that tightly couples these layers via a game-theoretic Model Predictive Controller (MPC) that uses a novel interactive multi-agent neural network policy as part of its predictive model. In our setting, the MPC planner considers all the surrounding agents by informing the multi-agent policy with the planned state sequence. Fundamental to the success of our method is the design of a novel multi-agent policy network that can steer a vehicle given the state of the surrounding agents and the map information. The policy network is trained implicitly with ground-truth observation data using backpropagation through time and a differentiable dynamics model to roll out the trajectory forward in time. Finally, we show that our multi-agent policy network learns to drive while interacting with the environment, and, when combined with the game-theoretic MPC planner, can successfully generate interactive behaviors.

Motivation & Objective

• Motivate coupling prediction and planning to account for two-way interactions among agents.
• Introduce a multi-agent interactive prediction (IMAP) policy that reacts to planned ego trajectories.
• Develop a model-based imitation learning approach to train the IMAP policy with differentiable dynamics.
• Incorporate map information and agent interactions to improve reactive decision making.
• Demonstrate interactive behavior generation on large-scale motion datasets.

Proposed method

• Formulate an abstract multi-agent policy that maps states, hidden states, and a map to actions using a differentiable unicycle dynamics model.
• Train the policy with model-based imitation learning using backpropagation through time and an imitation loss on state trajectories.
• Structure the IMAP policy with three interaction modules: intention (attention over agents), physical (graph neural network over states), and map (VectorNet-based map encoding with cross-attention).
• Use a recurrent GRU backbone to fuse interaction embeddings and produce actions as a squashed Gaussian for the differentiable dynamics.
• Apply closed-loop training to encourage reactive behavior rather than mere extrapolation, enabling best-response like planning in a game-theoretic MPC setting.
• Leverage a differentiable dynamics model to roll out trajectories and train end-to-end within an interactive prediction-planning framework.

Experimental results

Research questions

• RQ1 Can a learned multi-agent policy embedded in an MPC planner capture interactive ego and other agents behaviors without explicit reward function learning?
• RQ2 Does incorporating intention, physical, and map interactions in an IMAP policy improve reactive prediction and planning performance on real-world datasets?
• RQ3 Can model-based imitation learning with differentiable dynamics enable stable, interactive trajectory generation for multiple agents?
• RQ4 How does the interactive prediction-planning module perform when conditioned on HD-Map information and multi-agent interactions?

Key findings

• An interactive prediction-planning module combining prediction and game-theoretic planning is feasible and can generate interactive behaviors.
• A novel IMAP policy can learn to drive and interact with the environment using model-based imitation learning with differentiable dynamics.
• Three interaction channels—intention, physical, and map interactions—are integrated via GRU, GNN, and VectorNet components to produce actionable trajectories.
• The policy demonstrates reactive behavior on large-scale motion datasets Lyft Level 5 and Waymo Open Motion Dataset.
• Training uses closed-loop, teacher-forcing-inspired strategy to avoid excessive extrapolation and promote collision avoidance and lane following skills.