[Paper Review] From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting

Karttikeya Mangalam, Yang An+2 more|arXiv (Cornell University)|Dec 2, 2020

Autonomous Vehicle Technology and Safety32 references37 citations

TL;DR

This paper proposes Y-net, a scene-compliant trajectory forecasting model that factorizes multimodal uncertainty into epistemic (long-term goals) and aleatoric (intermediate waypoints and paths) components. By modeling goal distributions and path distributions hierarchically, Y-net achieves state-of-the-art performance on both short-term (SDD, ETH/UCY) and a novel long-term (up to 60 seconds) forecasting benchmark, improving SOTA by 51.9% on FDE and 77.1% on ADE in long-horizon settings.

ABSTRACT

Human trajectory forecasting is an inherently multi-modal problem. Uncertainty in future trajectories stems from two sources: (a) sources that are known to the agent but unknown to the model, such as long term goals and (b)sources that are unknown to both the agent & the model, such as intent of other agents & irreducible randomness indecisions. We propose to factorize this uncertainty into its epistemic & aleatoric sources. We model the epistemic un-certainty through multimodality in long term goals and the aleatoric uncertainty through multimodality in waypoints& paths. To exemplify this dichotomy, we also propose a novel long term trajectory forecasting setting, with prediction horizons upto a minute, an order of magnitude longer than prior works. Finally, we presentY-net, a scene com-pliant trajectory forecasting network that exploits the pro-posed epistemic & aleatoric structure for diverse trajectory predictions across long prediction horizons.Y-net significantly improves previous state-of-the-art performance on both (a) The well studied short prediction horizon settings on the Stanford Drone & ETH/UCY datasets and (b) The proposed long prediction horizon setting on the re-purposed Stanford Drone & Intersection Drone datasets.

Motivation & Objective

• To address the inherent multimodality in human trajectory forecasting by distinguishing between epistemic uncertainty (unknown long-term goals) and aleatoric uncertainty (unpredictable path variations).
• To propose a new long-term trajectory forecasting benchmark with prediction horizons up to 60 seconds, an order of magnitude longer than prior work.
• To design a scene-compliant deep learning model, Y-net, that explicitly models both goal and path multimodality for diverse and accurate long-horizon predictions.
• To demonstrate that factorizing uncertainty improves performance, especially in long-term forecasting where aleatoric variability dominates.

Proposed method

• Y-net uses a U-net-based architecture to predict explicit probability heatmaps for epistemic uncertainty (final goals) and aleatoric uncertainty (intermediate waypoints and full trajectories).
• The model estimates a multimodal distribution over long-term goals (Ke) and, conditioned on each sampled goal, predicts a multimodal distribution over intermediate waypoints and paths (Ka).
• It employs scene-conditional attention and graph-based message passing to incorporate scene semantics into trajectory predictions.
• The model uses a two-stage sampling strategy: first sampling goals from the estimated goal distribution, then sampling paths conditioned on each goal.
• A test-time sampling trick (TTST) is introduced to improve trajectory diversity and accuracy during inference.
• The framework supports both short-term and long-term forecasting by conditioning on different temporal horizons.

Experimental results

Research questions

• RQ1 How does factorizing epistemic and aleatoric uncertainty improve long-term human trajectory forecasting?
• RQ2 Can a unified model effectively predict diverse, scene-compliant trajectories over extended horizons (up to 60 seconds)?
• RQ3 How does the performance of short-term models degrade when applied recursively to long-term forecasting?
• RQ4 What is the impact of increasing path multimodality (Ka) and goal multimodality (Ke) on prediction accuracy?
• RQ5 Does explicit modeling of goal and path uncertainty outperform joint modeling of multimodality?

Key findings

• Y-net improves state-of-the-art performance on the Stanford Drone dataset by 34.0% on ADE and 51.9% on FDE in short-term forecasting (tf = 5s).
• On the ETH/UCY benchmark, Y-net achieves a 5.6% improvement in ADE and a 51.9% improvement in FDE over previous SOTA.
• In the proposed long-term forecasting setting (tf = 30s), Y-net improves ADE by 50.6% and FDE by 77.1% over PECNet on the Stanford Drone dataset.
• On the Intersection Drone dataset, Y-net reduces ADE by 35.0% and FDE by 55.9% compared to PECNet in long-term forecasting.
• The test-time sampling trick (TTST) reduces ADE by 9.1% on SDD and 18.5% on InD, and FDE by 30.4% and 35.0% respectively, showing its effectiveness in improving diversity and accuracy.
• Varying Ka (path multimodality) consistently improves ADE, and even with 20× more samples, PECNet underperforms Y-net at Ke = 20, proving the advantage of factorized uncertainty modeling.