[Paper Review] Learning to Remember Patterns: Pattern Matching Memory Networks for Traffic Forecasting

Hyunwook Lee, Seungmin Jin+3 more|arXiv (Cornell University)|Oct 20, 2021

Traffic Prediction and Management TechniquesEngineering18 references18 citations

TL;DR

This paper proposes Pattern-Matching Memory Networks (PM-MemNet), a novel traffic forecasting model that treats forecasting as a pattern-matching task using a key-value memory structure. By extracting and clustering representative traffic patterns and employing a graph convolutional memory network (GCMem), PM-MemNet achieves state-of-the-art performance, especially in long-term and rapidly changing traffic conditions, outperforming models like Graph WaveNet.

ABSTRACT

Traffic forecasting is a challenging problem due to complex road networks and sudden speed changes caused by various events on roads. A number of models have been proposed to solve this challenging problem with a focus on learning spatio-temporal dependencies of roads. In this work, we propose a new perspective of converting the forecasting problem into a pattern matching task, assuming that large data can be represented by a set of patterns. To evaluate the validness of the new perspective, we design a novel traffic forecasting model, called Pattern-Matching Memory Networks (PM-MemNet), which learns to match input data to the representative patterns with a key-value memory structure. We first extract and cluster representative traffic patterns, which serve as keys in the memory. Then via matching the extracted keys and inputs, PM-MemNet acquires necessary information of existing traffic patterns from the memory and uses it for forecasting. To model spatio-temporal correlation of traffic, we proposed novel memory architecture GCMem, which integrates attention and graph convolution for memory enhancement. The experiment results indicate that PM-MemNet is more accurate than state-of-the-art models, such as Graph WaveNet with higher responsiveness. We also present a qualitative analysis result, describing how PM-MemNet works and achieves its higher accuracy when road speed rapidly changes.

Motivation & Objective

• To address the challenge of accurately forecasting traffic under sudden changes and complex spatio-temporal dependencies.
• To explore whether traffic forecasting can be effectively framed as a pattern-matching task using representative historical patterns.
• To design a memory-augmented architecture that captures both spatial and temporal dependencies in road networks.
• To evaluate the performance of PM-MemNet against state-of-the-art models, particularly in long-term and abrupt change scenarios.
• To investigate the impact of memory size and architecture on forecasting accuracy and model robustness.

Proposed method

• Representative traffic patterns are extracted from historical data using clustering to form the key space in a memory structure.
• A novel graph convolutional memory network (GCMem) integrates attention and graph convolution to model spatio-temporal correlations in the memory.
• The encoder uses stacked GCMem layers with temporal embedding to generate contextual representations via pattern memorization.
• The decoder employs a gated recurrent unit (GRU) with GCMem to generate future traffic states based on matched patterns.
• Cosine similarity is used to match input sequences to the most relevant representative patterns in the memory.
• The model is trained end-to-end with a reconstruction loss, optimizing pattern matching and forecasting accuracy.

Experimental results

Research questions

• RQ1 Can traffic forecasting be effectively reformulated as a pattern-matching task using a learned memory of representative patterns?
• RQ2 How does the integration of graph convolution and attention in GCMem improve modeling of spatio-temporal dependencies in traffic data?
• RQ3 What is the optimal number of representative patterns for balancing model performance and memory efficiency?
• RQ4 How does PM-MemNet perform under abrupt traffic changes compared to existing models?
• RQ5 To what extent does the memory architecture influence long-term forecasting accuracy?

Key findings

• PM-MemNet outperforms Graph WaveNet and other state-of-the-art models on both METR-LA and NAVER-Seoul datasets, especially in long-term forecasting.
• The model achieves the best performance with 100 representative patterns, indicating that larger memory sizes do not always improve accuracy.
• A single-layer GCMem (PM-MemNet w/L=1) already delivers strong performance, showing the model's robustness and potential for lightweight deployment.
• The GRU-based decoder outperforms the CNN-based decoder in long-term prediction, and the RNN decoder shows higher precision than standard RNN models.
• Qualitative analysis confirms that PM-MemNet responds more quickly and accurately to abrupt speed changes, such as during rush hour or accidents.
• The ablation study confirms that GCMem is a robust architecture for modeling long-term dependencies in traffic sequences.