[Paper Review] Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance

Feixiang He, Yuanhang Xiang+2 more|arXiv (Cornell University)|Apr 29, 2020

Anomaly Detection Techniques and Applications37 references4 citations

TL;DR

This paper proposes a non-parametric Bayesian framework, Triplet-HDP with Chinese Restaurant Franchise League inference, to automatically decompose noisy, high-dimensional crowd trajectory data into interpretable modes capturing spatial, temporal, and dynamic patterns. The method enables unsupervised scene semantics discovery, enabling visualization, quantitative simulation comparison, and guidance—demonstrated effectively across diverse real-world datasets with up to 1000 trajectories and multi-day durations.

ABSTRACT

Crowd simulation is a central topic in several fields including graphics. To achieve high-fidelity simulations, data has been increasingly relied upon for analysis and simulation guidance. However, the information in real-world data is often noisy, mixed and unstructured, making it difficult for effective analysis, therefore has not been fully utilized. With the fast-growing volume of crowd data, such a bottleneck needs to be addressed. In this paper, we propose a new framework which comprehensively tackles this problem. It centers at an unsupervised method for analysis. The method takes as input raw and noisy data with highly mixed multi-dimensional (space, time and dynamics) information, and automatically structure it by learning the correlations among these dimensions. The dimensions together with their correlations fully describe the scene semantics which consists of recurring activity patterns in a scene, manifested as space flows with temporal and dynamics profiles. The effectiveness and robustness of the analysis have been tested on datasets with great variations in volume, duration, environment and crowd dynamics. Based on the analysis, new methods for data visualization, simulation evaluation and simulation guidance are also proposed. Together, our framework establishes a highly automated pipeline from raw data to crowd analysis, comparison and simulation guidance. Extensive experiments and evaluations have been conducted to show the flexibility, versatility and intuitiveness of our framework.

Motivation & Objective

• Address the challenge of analyzing large-scale, noisy, unstructured crowd trajectory data with mixed spatial, temporal, and dynamic information.
• Overcome limitations of existing methods that treat space, time, and dynamics separately or assume uniform motion randomness.
• Develop an unsupervised framework that automatically discovers recurring activity patterns (modes) from raw data without manual labeling.
• Enable flexible, interpretable visualization, quantitative evaluation of simulated crowds, and automated simulation guidance based on learned semantics.
• Support scalability to massive datasets (e.g., tens of thousands of people over multiple days) with fragmented or incomplete trajectory observations.

Proposed method

• Proposes a novel non-parametric Bayesian model, Triplet-HDP (THDP), which jointly models space, time, and speed dimensions using Hierarchical Dirichlet Processes to capture cross-dimensional correlations.
• Introduces a new inference algorithm, Chinese Restaurant Franchise League (CRFL), which enables efficient, joint sampling across the three HDPs by modeling the many-to-many associations between modes in different dimensions.
• Uses a two-phase inference strategy: first, a burn-in phase with only the Space-HDP to stabilize initial clustering; then full CRFL on the complete THDP to accelerate mixing and convergence.
• Applies space discretization (40×40 to 120×120 grids) and velocity estimation via perspective projection or direct time-differencing to normalize trajectory data.
• Employs MCMC sampling to avoid local minima, ensuring robust inference despite complex, high-dimensional data structures.
• Leverages the discovered modes as a basis for simulation guidance, where each mode represents a unique flow with specific spatial paths, temporal profiles, and speed preferences.

Experimental results

Research questions

• RQ1 How can we automatically extract structured, interpretable scene semantics from large-scale, noisy, unstructured crowd trajectory data?
• RQ2 Can we jointly model spatial, temporal, and dynamic dimensions of crowd behavior to capture recurring activity patterns without manual labeling?
• RQ3 How can the discovered modes be used to enable quantitative, dimension-specific evaluation of simulated crowds against real data?
• RQ4 To what extent can the framework support automated simulation guidance that captures heterogeneous motion randomness?
• RQ5 Can the framework scale effectively to real-world datasets with high volume, long duration, and fragmented observations?

Key findings

• The framework successfully decomposed crowd scenes into meaningful modes: 25 space, 5 time, and 7 speed modes in Forum; 13, 6, and 6 in CarPark; 16, 3, and 4 in TrainStation.
• Training completed in 85.1, 11.5, and 7.8 minutes on a standard PC for Forum, CarPark, and TrainStation, respectively, demonstrating scalability.
• The method achieved robust convergence using CRFL with a burn-in phase, outperforming independent HDPs that failed to preserve cross-dimensional associations.
• The framework enabled intuitive visualization of crowd semantics, revealing recurring flows such as entrances, exits, and information desks with temporal and speed profiles.
• Quantitative evaluation metrics derived from the modes allowed holistic and dimension-specific comparison between real and simulated crowds.
• Simulation guidance using the learned modes successfully captured heterogeneous motion randomness, avoiding the oversimplification of uniform or least-effort models.