[Paper Review] TensorAnalyzer: Identification of Urban Patterns in Big Cities using Non-Negative Tensor Factorization

Jaqueline Insaurriaga Silveira, Germain Garcia-Zanabria+4 more|arXiv (Cornell University)|Oct 6, 2022

Human Mobility and Location-Based Analysis4 citations

TL;DR

This paper proposes TensorAnalyzer, a non-negative tensor factorization-based framework that identifies urban patterns from multi-source geospatial data, outperforming traditional clustering methods like k-means and AHC in detecting meaningful spatial correlations. The method effectively reveals relationships between crime rates, socio-economic factors, and student performance in São Paulo, demonstrating superior clustering quality and robustness to noise.

ABSTRACT

Extracting relevant urban patterns from multiple data sources can be difficult using classical clustering algorithms since we have to make a suitable setup of the hyperparameters of the algorithms and deal with outliers. It should be addressed correctly to help urban planners in the decision-making process for the further development of a big city. For instance, experts' main interest in criminology is comprehending the relationship between crimes and the socio-economic characteristics at specific georeferenced locations. In addition, the classical clustering algorithms take little notice of the intricate spatial correlations in georeferenced data sources. This paper presents a new approach to detecting the most relevant urban patterns from multiple data sources based on tensor decomposition. Compared to classical methods, the proposed approach's performance is attested to validate the identified patterns' quality. The result indicates that the approach can effectively identify functional patterns to characterize the data set for further analysis in achieving good clustering quality. Furthermore, we developed a generic framework named TensorAnalyzer, where the effectiveness and usefulness of the proposed methodology are tested by a set of experiments and a real-world case study showing the relationship between the crime events around schools and students performance and other variables involved in the analysis.

Motivation & Objective

• To address the limitations of classical clustering in handling noisy, multi-source geospatial data with complex spatial correlations.
• To develop a robust, low-hyperparameter methodology for identifying functional urban patterns around target locations such as schools.
• To create a visualization-assisted tool that enables domain experts to explore and interpret urban patterns interactively.
• To validate the approach using real-world data from São Paulo, particularly the relationship between crime rates and student performance in public and private schools.
• To demonstrate the framework’s effectiveness through quantitative and qualitative experiments on synthetic and real datasets.

Proposed method

• Models multi-source geospatial data (e.g., crime, socio-economic, infrastructure) as a 3D tensor with spatial, temporal, and feature modes.
• Applies Non-Negative Tucker Decomposition (NTD) to extract low-rank, non-negative components representing latent urban patterns.
• Uses the Tucker decomposition to preserve spatial correlations and reduce noise, enhancing pattern interpretability.
• Employs a visualization interface (TensorAnalyzer) with map and pattern views to explore patterns around target locations like schools.
• Preprocesses data via tensor construction and applies NTD once as a pre-processing step, enabling fast interactive analysis.
• Incorporates expert knowledge to guide parameter selection (e.g., mode ranks, number of clusters), with plans to automate this via Fisher score in future work.

Experimental results

Research questions

• RQ1 Can non-negative tensor factorization effectively identify meaningful urban patterns from multi-source geospatial data while preserving spatial correlations?
• RQ2 How does the proposed method compare to traditional clustering algorithms (k-means, AHC) in terms of clustering quality and robustness to noise?
• RQ3 What is the relationship between urban crime patterns and student performance in schools across different socio-economic zones in São Paulo?
• RQ4 Can the TensorAnalyzer framework support interactive, expert-driven exploration of urban patterns around specific target locations?
• RQ5 How do temporal and spatial patterns of crime vary across different urban zones, especially in recreational and school areas?

Key findings

• TensorAnalyzer outperformed k-means and AHC across all evaluation metrics in synthetic and real data experiments, demonstrating higher clustering quality.
• The pattern M3 in the public school case study exhibited the highest homicide rate (584 incidents) and the worst student performance (ESEDIY: 5.36, ISPIFY: 4.17), indicating a strong inverse correlation between crime and academic outcomes.
• In recreational areas, passerby robberies were most frequent in the evening, with high activity observed across all patterns, suggesting elevated risk during nighttime hours.
• The computational cost of NTD was minimal (12 seconds for all patterns in the case study), enabling interactive analysis for urban planners and security experts.
• The framework successfully revealed that high-crime, low-performance zones were concentrated in poor neighborhoods with limited urban infrastructure.
• The system’s visualization components (map view, patterns view, control menu) enabled domain experts to explore and interpret complex urban patterns effectively.