[Paper Review] GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds

Tharindu Fernando, Simon Denman+2 more|arXiv (Cornell University)|Dec 18, 2018

Video Surveillance and Tracking Methods9 references4 citations

TL;DR

This paper proposes GD-GAN, a generative adversarial network framework that jointly predicts pedestrian trajectories and detects social groups in crowds using unsupervised deep feature learning. By leveraging spatio-temporal context and sparsity-regularized GAN training, the model learns task-specific representations that enable accurate, transferable group detection without hand-labeled data, outperforming state-of-the-art methods on multiple benchmarks.

ABSTRACT

This paper presents a novel deep learning framework for human trajectory prediction and detecting social group membership in crowds. We introduce a generative adversarial pipeline which preserves the spatio-temporal structure of the pedestrian's neighbourhood, enabling us to extract relevant attributes describing their social identity. We formulate the group detection task as an unsupervised learning problem, obviating the need for supervised learning of group memberships via hand labeled databases, allowing us to directly employ the proposed framework in different surveillance settings. We evaluate the proposed trajectory prediction and group detection frameworks on multiple public benchmarks, and for both tasks the proposed method demonstrates its capability to better anticipate human sociological behaviour compared to the existing state-of-the-art methods.

Motivation & Objective

• To address the challenge of detecting social groups in crowds without relying on hand-labeled datasets for group membership.
• To learn rich, task-specific representations of pedestrian behavior that encode social identity and interaction patterns.
• To develop a unified deep learning framework that jointly optimizes trajectory prediction and group detection.
• To enable direct transferability across different surveillance environments by avoiding retraining for new scenes.
• To improve the modeling of complex human navigation by replacing handcrafted features with end-to-end learned representations.

Proposed method

• The framework uses a GAN-based architecture where the generator predicts future trajectories and the discriminator enforces realistic spatio-temporal patterns.
• A custom loss function is designed to guide the generator in learning features relevant to both trajectory forecasting and group detection.
• Sparsity regularization is applied to the generator’s hidden representations to enhance discriminative power for group clustering.
• t-SNE dimensionality reduction is applied to the generator’s context representations to extract compact, meaningful features for clustering.
• DBSCAN clustering is applied to the reduced features to detect pedestrian groups in an unsupervised manner.
• The model is trained end-to-end using observed trajectories from $T_{obs}$ to predict $T_{pred}$ future steps, with context features extracted from the generator's hidden state.

Experimental results

Research questions

• RQ1 Can a unified deep learning framework jointly improve trajectory prediction and unsupervised group detection in crowds?
• RQ2 Can sparsity-regularized GANs learn more discriminative pedestrian representations than standard supervised or GAN-based methods?
• RQ3 Does the proposed method achieve better group detection performance than state-of-the-art approaches without requiring hand-labeled group annotations?
• RQ4 To what extent do the learned features capture social interaction patterns such as proximity, alignment, and common goals?
• RQ5 How transferable is the framework across different surveillance scenes without retraining?

Key findings

• The proposed GD-GAN model achieved 79.2% F1-score on the CEB benchmark and 68.4% on Student-003 for group detection, outperforming all ablation variants.
• The ablation study showed that removing the GAN discriminator (GD-GAN / GAN) led to poor performance (73.6% P, 75.1% R), indicating the necessity of adversarial training.
• Adding $L_1$ sparsity regularization improved group detection accuracy over the standard cGAN baseline, demonstrating its role in enhancing feature discriminability.
• Combining GD-GAN features with handcrafted features (GD-GAN + hf) did not significantly improve performance, indicating the model’s features are already highly informative.
• The model achieved real-time inference, processing 100 trajectories with 30x2D points in 0.712 seconds on a single CPU core, showing strong time efficiency.
• Visualization of t-SNE projections confirmed that trajectories from the same group were embedded more tightly together, validating the model’s ability to encode social interaction patterns.