[Paper Review] Recursive Social Behavior Graph for Trajectory Prediction

Jianhua Sun, Qinhong Jiang+1 more|arXiv (Cornell University)|Apr 22, 2020

Autonomous Vehicle Technology and Safety44 references17 citations

TL;DR

This paper proposes Recursive Social Behavior Graph (RSBG), a novel graph-based method that models social interactions in human trajectory prediction using group-based annotations to capture distance-invariant relationships. By recursively updating pedestrian features through a graph convolutional network guided by RSBG, the model achieves 11.1% and 10.8% average improvements in ADE and FDE over state-of-the-art methods on ETH and UCY datasets.

ABSTRACT

Social interaction is an important topic in human trajectory prediction to generate plausible paths. In this paper, we present a novel insight of group-based social interaction model to explore relationships among pedestrians. We recursively extract social representations supervised by group-based annotations and formulate them into a social behavior graph, called Recursive Social Behavior Graph. Our recursive mechanism explores the representation power largely. Graph Convolutional Neural Network then is used to propagate social interaction information in such a graph. With the guidance of Recursive Social Behavior Graph, we surpass state-of-the-art method on ETH and UCY dataset for 11.1% in ADE and 10.8% in FDE in average, and successfully predict complex social behaviors.

Motivation & Objective

• To address the limitation of distance-based interaction modeling in human trajectory prediction, which fails to capture complex social behaviors like following or collision avoidance.
• To model social relationships beyond spatial proximity by introducing group-based annotations as supervision for learning latent social representations.
• To improve trajectory prediction accuracy by leveraging graph neural networks to propagate social interaction information through a recursively updated social behavior graph.
• To demonstrate that group-level social norms and long-range interactions can be effectively captured and utilized in deep learning frameworks for trajectory forecasting.

Proposed method

• Constructs a Recursive Social Behavior Graph (RSBG) where pedestrians are nodes and relational social representations (edges) encode group-level interaction patterns.
• Uses group annotations from expert-labeled data to supervise the learning of social representations, enabling the model to learn latent social relationships beyond Euclidean distance.
• Employs a recursive mechanism that alternately updates individual pedestrian features and social representations using message passing in the graph.
• Applies Graph Convolutional Networks (GCNs) to propagate and aggregate social interaction features across nodes in the RSBG for improved trajectory prediction.
• Utilizes an Exponential L2 Loss function with a hyperparameter γ to emphasize FDE (Final Distance Error) during training, improving final prediction accuracy.
• Integrates historical trajectory features as input to each node, enabling the model to consider temporal dynamics in social behavior.

Experimental results

Research questions

• RQ1 Can group-based annotations improve the modeling of complex, non-distance-based social interactions in trajectory prediction?
• RQ2 How does recursive feature refinement in a social behavior graph enhance prediction accuracy compared to static or distance-based methods?
• RQ3 To what extent can GCNs effectively propagate social interaction information through a dynamically updated graph structure?
• RQ4 Can the proposed method generalize to diverse social behaviors such as following, joining, and collision avoidance in complex scenes?
• RQ5 Does emphasizing FDE in the loss function lead to better final prediction performance compared to standard L2 loss?

Key findings

• The proposed RSBG model achieves a 11.1% average improvement in Average Displacement Error (ADE) and 10.8% improvement in Final Displacement Error (FDE) over state-of-the-art methods on the ETH and UCY benchmark datasets.
• The model successfully captures long-range social relationships, such as a pedestrian following someone far away, which distance-based models fail to detect.
• Visualization of RSBG edge weights confirms that the model learns consistent social norms, such as following behavior or collision avoidance, even when agents are not spatially close.
• The Exponential L2 Loss with γ=20 yields the best trade-off, reducing ADE and FDE by 4.0% and 4.8% respectively compared to standard L2 loss.
• Qualitative results show that RSBG generates more socially plausible trajectories, correctly predicting joining, following, and conflict-avoidance behaviors that other models miss.
• The recursive mechanism enables iterative refinement of both individual features and social representations, leading to more accurate and contextually consistent predictions.