[Paper Review] Proceedings of the 24th Computer Vision Winter Workshop

Haddad, Sirin, Wu, Meiqing+2 more|arXiv (Cornell University)|Feb 13, 2019

Autonomous Vehicle Technology and Safety26 references53 citations

TL;DR

This paper proposes a spatio-temporal graph-based LSTM with a dynamic attention mechanism that models both human-human and human-obstacle interactions in crowded scenes for pedestrian trajectory prediction. By incorporating static obstacles only when they are in close proximity and using a multi-head attention mechanism over spatial and temporal dimensions, the method reduces ADE by up to 55% and FDE by up to 61% over state-of-the-art approaches on benchmark datasets.

ABSTRACT

Pedestrian trajectory prediction is essential for collision avoidance in autonomous driving and robot navigation. However, predicting a pedestrian's trajectory in crowded environments is non-trivial as it is influenced by other pedestrians' motion and static structures that are present in the scene. Such human-human and human-space interactions lead to non-linearities in the trajectories. In this paper, we present a new spatio-temporal graph based Long Short-Term Memory (LSTM) network for predicting pedestrian trajectory in crowded environments, which takes into account the interaction with static (physical objects) and dynamic (other pedestrians) elements in the scene. Our results are based on two widely-used datasets to demonstrate that the proposed method outperforms the state-of-the-art approaches in human trajectory prediction. In particular, our method leads to a reduction in Average Displacement Error (ADE) and Final Displacement Error (FDE) of up to 55% and 61% respectively over state-of-the-art approaches.

Motivation & Objective

• To improve pedestrian trajectory prediction in crowded outdoor environments by modeling both dynamic (pedestrian) and static (obstacles) interactions.
• To address the limitations of prior models that either ignore static obstacles or apply global attention indiscriminately.
• To develop a more stable and accurate prediction model by dynamically incorporating static obstacles only when they are contextually relevant.
• To enhance trajectory prediction by explicitly modeling local interactions with obstacles and global interactions among pedestrians using a graph-based attention mechanism.

Proposed method

• Proposes a spatio-temporal graph that encodes both pedestrians and static obstacles as nodes, with edges representing spatial-temporal relationships.
• Introduces a dynamic graph construction strategy that includes static obstacles only when they are within a proximity threshold likely to affect pedestrian motion.
• Employs a multi-head attention mechanism over the graph to compute attention weights based on spatial and temporal features of pedestrians and obstacles.
• Uses a simplified multi-head attention mechanism inspired by [12] to aggregate influence from neighboring agents and obstacles across time.
• Trains the model using a negative log-likelihood loss over future trajectory steps, minimizing prediction error.
• Applies a variational inference framework to generate diverse, plausible future trajectories with uncertainty modeling.

Experimental results

Research questions

• RQ1 Can a spatio-temporal graph-based model that dynamically incorporates static obstacles improve trajectory prediction accuracy in crowded scenes?
• RQ2 How does modeling both global pedestrian interactions and local obstacle interactions affect prediction performance compared to models that consider only one or the other?
• RQ3 To what extent does a graph-based attention mechanism that weighs influence from nearby pedestrians and obstacles reduce displacement errors in trajectory prediction?
• RQ4 Does the proposed method generalize across diverse urban environments with varying crowd density and obstacle presence?

Key findings

• The H-H-O model achieves a 55% reduction in Average Displacement Error (ADE) and a 61% reduction in Final Displacement Error (FDE) compared to state-of-the-art methods on average across datasets.
• On the Hotel dataset, the model reduces FDE by 93% compared to Social GAN-20V, demonstrating strong performance in scenes with dense static obstacles like lamp posts and trees.
• On the ETH dataset, the model reduces FDE by 89% compared to Social-LSTM, highlighting improved collision avoidance in high-density pedestrian interactions.
• The minimum FDE achieved by the H-H-O model is 0.11m in the Hotel set, significantly lower than the 0.80m minimum FDE of Social-LSTM and 0.84m of SGAN-20V.
• Qualitative results show that the H-H-O model correctly predicts avoidance behaviors around benches and shop entrances, while baseline models fail to respect physical obstacles.
• The model’s predictions are more consistent with ground-truth trajectories, especially at the final prediction step, due to better modeling of environmental constraints.