[Paper Review] Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans

Nachiket Deo, Mohan M. Trivedi|arXiv (Cornell University)|Jan 3, 2020

Autonomous Vehicle Technology and Safety47 references114 citations

TL;DR

P2T proposes Plans-to-Trajectories: forecasting multimodal pedestrian and vehicle trajectories in unknown environments by inferring grid-based plans via MaxEnt IRL and generating continuous trajectories with an attention-based decoder conditioned on those plans.

ABSTRACT

We address the problem of forecasting pedestrian and vehicle trajectories in unknown environments, conditioned on their past motion and scene structure. Trajectory forecasting is a challenging problem due to the large variation in scene structure and the multimodal distribution of future trajectories. Unlike prior approaches that directly learn one-to-many mappings from observed context to multiple future trajectories, we propose to condition trajectory forecasts on plans sampled from a grid based policy learned using maximum entropy inverse reinforcement learning (MaxEnt IRL). We reformulate MaxEnt IRL to allow the policy to jointly infer plausible agent goals, and paths to those goals on a coarse 2-D grid defined over the scene. We propose an attention based trajectory generator that generates continuous valued future trajectories conditioned on state sequences sampled from the MaxEnt policy. Quantitative and qualitative evaluation on the publicly available Stanford drone and NuScenes datasets shows that our model generates trajectories that are diverse, representing the multimodal predictive distribution, and precise, conforming to the underlying scene structure over long prediction horizons.

Motivation & Objective

• Forecast trajectories for pedestrians and vehicles in unknown environments using past motion and scene layout.
• Infer plausible goals and paths on a coarse 2-D grid without predefined goals.
• Generate continuous trajectories conditioned on sampled grid plans with an interpretable plan representation.
• Produce diverse yet scene-compliant trajectories and provide a compact set of predictions for downstream planning.

Proposed method

• Reformulate MaxEnt IRL to jointly infer transient path rewards and terminal goal rewards.
• Learn a reward model (CNN-based) that maps local scene patches to path and goal rewards at grid cells.
• Use a non-goal-conditioned MaxEnt policy to sample multimodal grid-based plans to potential goals.
• Construct an attention-based trajectory generator that maps sampled plans and motion history to continuous future trajectories.
• Train the trajectory generator with an encoder–decoder (GRU and BiGRU) plus soft attention to produce trajectories conditioned on plans.
• Cluster sampled trajectories into K representative futures for downstream planning.

Experimental results

Research questions

• RQ1 Can grid-based MaxEnt IRL infer plausible, multimodal goals and paths in unknown scenes without pre-specified endpoints?
• RQ2 Do trajectories conditioned on sampled grid plans better conform to scene structure and demonstrate long-horizon accuracy compared to prior multimodal approaches?
• RQ3 Can an attention-based trajectory generator conditioned on plans produce diverse yet precise futures suitable for downstream planning in autonomous systems?

Key findings

• The model generates diverse trajectories that conform to the underlying scene over long horizons.
• P2T achieves strong sample quality metrics and competitive or state-of-the-art results on several evaluated metrics across Stanford Drone and NuScenes datasets.
• The approach improves precision while maintaining diversity, addressing the typical recall-precision trade-off in multimodal forecasting.
• Providing K clustered trajectories yields a compact, planner-friendly representation without retraining the model for different K values.