[Paper Review] Deep Learning for Spatio-Temporal Data Mining: A Survey

Senzhang Wang, Jiannong Cao+1 more|arXiv (Cornell University)|Jun 11, 2019

Traffic Prediction and Management Techniques181 references33 citations

TL;DR

A comprehensive survey of how deep learning models (CNN, RNN, GraphCNN, etc.) are applied to spatio-temporal data mining, detailing data types, representations, frameworks, applications, and future directions.

ABSTRACT

With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal datasets increase rapidly, traditional data mining methods, especially statistics based methods for dealing with such data are becoming overwhelmed. Recently, with the advances of deep learning techniques, deep leaning models such as convolutional neural network (CNN) and recurrent neural network (RNN) have enjoyed considerable success in various machine learning tasks due to their powerful hierarchical feature learning ability in both spatial and temporal domains, and have been widely applied in various spatio-temporal data mining (STDM) tasks such as predictive learning, representation learning, anomaly detection and classification. In this paper, we provide a comprehensive survey on recent progress in applying deep learning techniques for STDM. We first categorize the types of spatio-temporal data and briefly introduce the popular deep learning models that are used in STDM. Then a framework is introduced to show a general pipeline of the utilization of deep learning models for STDM. Next we classify existing literatures based on the types of ST data, the data mining tasks, and the deep learning models, followed by the applications of deep learning for STDM in different domains including transportation, climate science, human mobility, location based social network, crime analysis, and neuroscience. Finally, we conclude the limitations of current research and point out future research directions.

Motivation & Objective

• Categorize spatio-temporal data types and their representations.
• Present a general framework for applying deep learning to STDM.
• Review how different DL models map to ST data representations.
• Survey DL-based STDM applications across transportation, climate science, human mobility, and more.
• Identify limitations and propose future research directions.

Proposed method

• Introduce a general pipeline for STDM using deep learning: data instance construction, data representation, model selection/design, and problem addressing.
• Categorize ST data into event, trajectory, point reference, raster, and video types and map them to data representations (sequence, graph, 2D matrix, 3D tensor).
• Survey DL models (CNN, GraphCNN, RNN/LSTM/GRU, ConvLSTM, Seq2Seq, AE/SAE, RBM/DBN, GAN, etc.) and their suitability for different data representations.
• Discuss STDM tasks such as prediction, classification, representation learning, and anomaly detection in a unified framework.
• Provide guidance on preprocessing steps and hybrid model design to capture spatial and temporal correlations.
• Highlight how data representations influence model choice (e.g., sequences for RNNs, matrices/tensors for CNNs, graphs for GraphCNN).
• Summarize representative works and domain-specific applications.

Experimental results

Research questions

• RQ1 What types of spatio-temporal data exist and how can they be effectively represented for deep learning?
• RQ2 Which deep learning models are best suited for each ST data type and representation in common STDM tasks?
• RQ3 How can a general framework guide the application of DL to STDM across domains?
• RQ4 What are the key limitations of current DL-STDM approaches and future research directions?

Key findings

• There has been a rapid growth in DL for STDM papers since 2016, with a notable increase in 2018.
• CNN and RNN-based architectures (including GraphCNN and ConvLSTM) are the most widely used for processing spatial maps, rasters, trajectories, and time series.
• A general, structured framework is proposed to guide STDM with DL, from data instantiation and representation to model design and task addressing.
• ST data are categorized into event, trajectory, point reference, raster, and video types, with corresponding data representations and DL model mappings.
• Hybrid models (e.g., CNN+RNN) are common to capture both spatial and temporal dependencies.
• The survey covers applications across transportation, climate science, human mobility, location-based social networks, crime analysis, and neuroscience, and discusses limitations and future directions.