[Paper Review] OCTNet: Trajectory Generation in New Environments from Past Experiences

Weiming Zhi, Tin Lai+3 more|arXiv (Cornell University)|Sep 25, 2019

Autonomous Vehicle Technology and Safety25 references4 citations

TL;DR

OCTNet is a probabilistic generative model that learns to produce multi-modal, map-aware motion trajectories in new environments by generalizing from past trajectories observed in different environments. It encodes occupancy maps as feature vectors and uses neural networks to model conditional distributions over trajectory parameters, enabling high-quality, valid trajectory generation without rejection sampling, outperforming GANs and CVAEs in trajectory fidelity and map adherence.

ABSTRACT

Being able to safely operate for extended periods of time in dynamic environments is a critical capability for autonomous systems. This generally involves the prediction and understanding of motion patterns of dynamic entities, such as vehicles and people, in the surroundings. Many motion prediction methods in the literature can learn a function, mapping position and time to potential trajectories taken by people or other dynamic entities. However, these predictions depend only on previously observed trajectories, and do not explicitly take into consideration the environment. Trends of motion obtained in one environment are typically specific to that environment, and are not used to better predict motion in other environments. In this paper, we address the problem of generating likely motion dynamics conditioned on the environment, represented as an occupancy map. We introduce the Occupancy Conditional Trajectory Network (OCTNet) framework, capable of generalising the previously observed motion in known environments, to generate trajectories in new environments where no observations of motion has not been observed. OCTNet encodes trajectories as a fixed-sized vector of parameters and utilises neural networks to learn conditional distributions over parameters. We empirically demonstrate our method's ability to generate complex multi-modal trajectory patterns in different environments.

Motivation & Objective

• To enable autonomous agents to generate plausible motion trajectories in new environments where no motion data has been observed.
• To develop a map-aware trajectory generation model that generalizes from past experiences in different environments.
• To model the probabilistic and multi-modal nature of human and dynamic entity motion in complex environments.
• To generate continuous, queryable trajectories at arbitrary temporal resolutions without relying on forward sampling or error accumulation.
• To improve generalization by encoding occupancy maps as similarity-based feature vectors and conditioning trajectory generation on them.

Proposed method

• OCTNet represents trajectories as fixed-size vectors of basis function coefficients (w), enabling compact and differentiable trajectory encoding.
• It uses a neural network to learn a conditional distribution p(w | φ), where φ is a learned embedding of the occupancy map, capturing environmental structure.
• The model employs a mixture density network (MDN) to model multi-modal trajectory distributions, allowing sampling of diverse, realistic motion patterns.
• It evaluates two prior distributions—normal and Laplace—over the elements of w, with Laplace yielding better performance in experiments.
• Trajectories are reconstructed from sampled w vectors using kernel-based interpolation, enabling continuous, resolution-flexible trajectory queries.
• The model is trained end-to-end using maximum likelihood estimation, with occupancy maps encoded via a Siamese-like network to capture map similarities.

Experimental results

Research questions

• RQ1 Can a generative model learn to produce valid, multi-modal motion trajectories in new environments using only past trajectory data from different environments?
• RQ2 How well can a map-conditional model generalize to unseen environments without requiring motion data in those environments?
• RQ3 What is the impact of different prior distributions on the quality and diversity of generated trajectories?
• RQ4 How does the model compare to standard generative models like GANs and CVAEs in terms of trajectory validity and map adherence?
• RQ5 Can the model generate continuous trajectories at arbitrary time resolutions without error accumulation from forward sampling?

Key findings

• OCTNet generates high-quality, valid trajectories in unseen environments without rejection sampling, with approximately 33% of sampled trajectories being valid, compared to significantly lower success rates in GANs and CVAEs.
• The Laplace prior over trajectory parameters outperformed the normal prior, achieving a Hausdorff distance of 1.86 and Frechet distance of 2.00, indicating superior trajectory fidelity.
• OCTNet outperformed GANs and CVAEs across all metrics: Hausdorff (11.79 vs. 1.86), Frechet (16.66 vs. 2.00), and DTW (1147.40 vs. 93.10), demonstrating superior trajectory quality and map compliance.
• The use of a map embedding φ that captures similarity across environments enables effective generalization even with limited training data on occupancy maps.
• The model successfully captures multi-modal motion patterns, as evidenced by diverse, environment-consistent trajectory outputs in unseen maps.
• The continuous trajectory representation via kernel-based interpolation allows for trajectory queries at arbitrary time points, supporting flexible downstream planning.