[Paper Review] LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction

Jiawei Jiang, Chengkai Han+3 more|arXiv (Cornell University)|Apr 27, 2023

Human Mobility and Location-Based Analysis4 citations

TL;DR

LibCity is an open-source, PyTorch-based library that provides a unified framework for urban spatial-temporal prediction, offering 55 standardized datasets and 65 pre-implemented models across 9 sub-tasks. It enables fair model comparison, simplifies model development via modular design and atomic file data format, and establishes a comprehensive benchmark for reproducible research in smart city applications.

ABSTRACT

As deep learning technology advances and more urban spatial-temporal data accumulates, an increasing number of deep learning models are being proposed to solve urban spatial-temporal prediction problems. However, there are limitations in the existing field, including open-source data being in various formats and difficult to use, few papers making their code and data openly available, and open-source models often using different frameworks and platforms, making comparisons challenging. A standardized framework is urgently needed to implement and evaluate these methods. To address these issues, we propose LibCity, an open-source library that offers researchers a credible experimental tool and a convenient development framework. In this library, we have reproduced 65 spatial-temporal prediction models and collected 55 spatial-temporal datasets, allowing researchers to conduct comprehensive experiments conveniently. By enabling fair model comparisons, designing a unified data storage format, and simplifying the process of developing new models, LibCity is poised to make significant contributions to the spatial-temporal prediction field.

Motivation & Objective

• Address the lack of standardized, reproducible frameworks for urban spatial-temporal prediction due to inconsistent data formats, limited code/data availability, and fragmented model implementations.
• Enable fair and comprehensive model comparison by providing a unified data storage format, standardized evaluation, and consistent hyperparameter settings.
• Simplify the development of new models through a modular, extensible framework with well-defined interfaces for configuration, data, model, evaluation, and execution.
• Establish a benchmark for spatial-temporal prediction by collecting 55 widely used datasets and reproducing 65 state-of-the-art models across macro, micro, and fundamental prediction tasks.
• Promote standardization and community adoption in urban AI research by supporting extensibility, automated hyperparameter tuning, and visualization tools.

Proposed method

• Design a five-module framework (Configuration, Data, Model, Evaluation, Execution) built on PyTorch to ensure modularity, reusability, and seamless integration across components.
• Introduce 'atomic files'—a generalized, extensible data format representing five core data types: geographical, user, relation, spatiotemporal dynamic, and external data—enabling consistent data handling across diverse sources.
• Implement a standardized batch input format for models, derived from atomic files, to streamline data loading and model training across different tasks.
• Support model development via a minimal interface: developers only need to implement three core interfaces (model definition, loss function, and forward pass), while LibCity manages training, evaluation, and hyperparameter tuning.
• Integrate automated hyperparameter tuning and a visualization platform to enhance usability and experimental transparency.
• Benchmark model performance across 55 datasets and 65 models, establishing a public leaderboard to guide future research.

Experimental results

Research questions

• RQ1 How can a unified, modular framework improve reproducibility and comparability in urban spatial-temporal prediction research?
• RQ2 To what extent can a standardized data format like atomic files reduce the complexity of handling heterogeneous urban spatiotemporal datasets?
• RQ3 Can a single library effectively support both macro-level (e.g., traffic flow) and micro-level (e.g., trajectory prediction) prediction tasks with consistent interfaces?
• RQ4 How does LibCity’s modular design facilitate the development and evaluation of new models compared to existing libraries?
• RQ5 What impact does standardization of data, models, and evaluation have on the scalability and community adoption of urban AI research?

Key findings

• LibCity supports 55 diverse urban spatial-temporal datasets and 65 pre-implemented models, covering 9 sub-tasks including traffic flow, speed, OD matrix, and trajectory prediction.
• The library’s modular design enables new model development with minimal code, requiring only three core interfaces, significantly reducing implementation overhead.
• LibCity’s atomic file format ensures consistent data representation and processing, minimizing data preparation effort across heterogeneous sources.
• The library has achieved 485 GitHub stars and 111 forks, indicating strong community adoption and active maintenance, with only 73 total issues and 0 open issues at the time of publication.
• Compared to existing tools like DL-Traff, DGCRN, and FOST, LibCity outperforms in model and dataset coverage, modularity, and extensibility.
• The comprehensive benchmarking framework enables fair, reproducible model evaluation and identifies promising research directions through a public performance leaderboard.