[Paper Review] TNT: Target-driveN Trajectory Prediction

Hang Zhao, Jiyang Gao+10 more|arXiv (Cornell University)|Aug 19, 2020

Autonomous Vehicle Technology and Safety38 references210 citations

TL;DR

TNT proposes a three-stage, target-driven framework for multimodal trajectory prediction that discretizes future targets, conditions motion on these targets, and scores trajectories to produce a compact set of likely futures; it achieves state-of-the-art results on multiple benchmarks.

ABSTRACT

Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and intrinsically multimodal. Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states. This leads to our target-driven trajectory prediction (TNT) framework. TNT has three stages which are trained end-to-end. It first predicts an agent's potential target states $T$ steps into the future, by encoding its interactions with the environment and the other agents. TNT then generates trajectory state sequences conditioned on targets. A final stage estimates trajectory likelihoods and a final compact set of trajectory predictions is selected. This is in contrast to previous work which models agent intents as latent variables, and relies on test-time sampling to generate diverse trajectories. We benchmark TNT on trajectory prediction of vehicles and pedestrians, where we outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.

Motivation & Objective

• Motivate robust multimodal trajectory prediction by explicitly modeling a discrete set of plausible future targets.
• Introduce a three-stage, end-to-end trainable framework that separates target prediction, target-conditioned motion, and trajectory scoring.
• Show that targets capture most long-horizon uncertainty and that motion given a target is effectively unimodal.

Proposed method

• Stage 1 (Target Prediction): predict a distribution over discrete future target locations using scene context; over-sample targets (e.g., N ~ 1000) and output top M targets with associated offsets; train with cross-entropy for target probability and Huber loss for offsets.
• Stage 2 (Target-conditioned Motion Estimation): for each target, predict a unimodal trajectory conditioned on the target and context; use a 2-layer MLP and teacher forcing during training.
• Stage 3 (Trajectory Scoring and Selection): score trajectories with a maximum-entropy model, learn to rank and select a diverse, compact set of K predictions using a cross-entropy loss against ground-truth-like scores; employ non-maximum suppression to remove near-duplicates.

Experimental results

Research questions

• RQ1 Can discretizing future targets capture the majority of multimodal uncertainty in trajectory prediction?
• RQ2 Does conditioning trajectory generation on discrete targets enable accurate, diverse yet compact predictions without relying on latent variable sampling at test time?
• RQ3 How does the TNT pipeline perform across driving and pedestrian datasets compared to state-of-the-art methods?
• RQ4 What is the impact of target sampling density on prediction accuracy and diversity?

Key findings

• TNT achieves state-of-the-art results on four benchmarks: Argoverse, INTERACTION, PAID, and SDD.
• Using target-driven conditioning yields strong recall and accuracy with a compact set of predictions (e.g., top M targets and K final trajectories).
• Dense target sampling improves performance up to a saturation point, with grid targets performing well for pedestrians.
• Target offset regression and the unimodal motion model per target significantly improve accuracy compared to variants without these components.
• Compared to latent-variable or anchor-based approaches, TNT provides interpretable intermediate outputs (targets) and competitive or superior predictive performance.
• A single TNT model reaches or exceeds performance of challenge winners on several datasets.