[Paper Review] Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data

Sergio Casas, Cole Gulino+2 more|arXiv (Cornell University)|Oct 18, 2019

Anomaly Detection Techniques and Applications50 references52 citations

TL;DR

SpAGNN introduces a spatially aware graph neural network that jointly performs object detection and probabilistic, relational motion forecasting for autonomous driving by modeling interactions among detected agents. It achieves state-of-the-art results on ATG4D and nuScenes.

ABSTRACT

In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.

Motivation & Objective

• Motivate and address relational behavior forecasting in autonomous driving by modeling interactions among agents.
• Develop a jointly trainable model that combines perception (detection) and social-forecasting (trajectory prediction).
• Leverage a spatially aware GNN inspired by Gaussian belief propagation to capture multi-agent interactions.
• Produce probabilistic trajectory predictions that quantify uncertainty at the trajectory level.

Proposed method

• Two-stage architecture: input from LiDAR and HD maps for object detection and RoI-based feature extraction for each actor.
• A fully connected directed interaction graph where each detected actor is a node."
• Spatially-aware graph neural network (SpAGNN) that unrolls message passing; messages are transformed by spatial coordinates to align with neighbor frames.
• Output distributions per actor: Gaussian for x,y trajectories and Von Mises for heading, with parameters updated via GNN message passing.
• End-to-end training with a multi-task objective including detection loss (classification and box regression) and negative log-likelihood for trajectory predictions.

Experimental results

Research questions

• RQ1 How can multi-agent interactions be effectively modeled to improve both detection and relational motion forecasting in autonomous driving?
• RQ2 Does a spatially-aware GNN that leverages local coordinate transformations improve probabilistic trajectory predictions and social compliance against baselines?
• RQ3 Can end-to-end training mitigate error propagation from perception to forecasting in joint detection-prediction frameworks?
• RQ4 What is the impact of using RoI-aligned actor features and spatial transformations on forecasting accuracy and uncertainty modeling?

Key findings

• SpAGNN improves detection and interaction-aware forecasting over state-of-the-art baselines on ATG4D and nuScenes.
• Incorporating spatially transformed messages and local coordinate systems reduces collision rates and improves centroid L2 and heading errors across multiple horizons.
• Rotated RoI Align-based per-actor features outperform simple feature indexing for RoI features.
• Spatial awareness (bounding boxes in global/global-relative coordinates) and distributional outputs (Gaussian + Von Mises) contribute to lower predictive entropy and more accurate forecasts.