[Paper Review] Learning Trajectory Prediction with Continuous Inverse Optimal Control via Langevin Sampling of Energy-Based Models.

Yifei Xu, Tianyang Zhao+3 more|arXiv (Cornell University)|Apr 10, 2019

Autonomous Vehicle Technology and Safety23 references10 citations

TL;DR

This paper proposes a model-based inverse optimal control method using Langevin sampling in energy-based models to predict vehicle trajectories in autonomous driving. By learning non-Markovian, neural-augmented cost functions from demonstrations, it achieves state-of-the-art prediction accuracy while incorporating kinematic constraints and scene context.

ABSTRACT

Autonomous driving is a challenging multiagent domain which requires optimizing complex, mixed cooperative-competitive interactions. Learning to predict contingent distributions over other vehicles' trajectories simplifies the problem, allowing approximate solutions by trajectory optimization with dynamic constraints. We take a model-based approach to prediction, in order to make use of structured prior knowledge of vehicle kinematics, and the assumption that other drivers plan trajectories to minimize an unknown cost function. We introduce a novel inverse optimal control (IOC) algorithm to learn other vehicles' cost functions in an energy-based generative model. Langevin Sampling, a Monte Carlo based sampling algorithm, is used to directly sample the control sequence. Our algorithm provides greater flexibility than standard IOC methods, and can learn higher-level, non-Markovian cost functions defined over entire trajectories. We extend weighted feature-based cost functions with neural networks to obtain NN-augmented cost functions, which combine the advantages of both model-based and model-free learning. Results show that model-based IOC can achieve state-of-the-art vehicle trajectory prediction accuracy, and naturally take scene information into account.

Motivation & Objective

• To improve trajectory prediction in autonomous driving by modeling complex cooperative-competitive interactions among vehicles.
• To learn unknown driver cost functions from observed trajectories using inverse optimal control.
• To enable flexible, non-Markovian cost function learning over entire trajectories, beyond standard Markovian assumptions.
• To integrate structured prior knowledge of vehicle kinematics with data-driven neural networks for improved generalization.
• To develop a sampling-based inference method that directly generates control sequences while respecting dynamic constraints.

Proposed method

• Uses an energy-based generative model to represent the cost function of driving behaviors.
• Applies Langevin sampling—a Monte Carlo method—to directly sample control sequences from the energy-based model.
• Introduces a novel inverse optimal control algorithm that learns cost functions from observed vehicle trajectories.
• Augments feature-based cost functions with neural networks to model complex, high-level driving preferences.
• Incorporates vehicle kinematic constraints as prior knowledge in the model structure.
• Optimizes the cost function using gradient-based learning while maintaining trajectory feasibility through sampling.

Experimental results

Research questions

• RQ1 Can inverse optimal control with Langevin sampling improve trajectory prediction accuracy in multiagent driving scenarios?
• RQ2 How well can the model learn non-Markovian cost functions that depend on entire trajectories rather than local states?
• RQ3 To what extent does combining neural networks with model-based priors enhance prediction performance?
• RQ4 Can the method naturally incorporate scene context and dynamic constraints in trajectory prediction?
• RQ5 How does the proposed approach compare to existing model-free and model-based baselines?

Key findings

• The proposed method achieves state-of-the-art trajectory prediction accuracy by leveraging structured priors and neural-augmented cost functions.
• Langevin sampling enables direct generation of feasible control sequences while respecting dynamic constraints.
• The model successfully learns higher-level, non-Markovian cost functions that depend on entire trajectories, improving prediction fidelity.
• Neural-augmented cost functions outperform traditional feature-based functions by capturing complex driving behaviors.
• The method naturally incorporates scene context and kinematic constraints, leading to more realistic and safe trajectory predictions.
• The approach demonstrates greater flexibility than standard inverse optimal control methods, especially in complex, mixed cooperative-competitive environments.