[Paper Review] Minimalist Traffic Prediction: Linear Layer Is All You Need

Wenying Duan, Hong Rao+2 more|arXiv (Cornell University)|Aug 20, 2023

Traffic Prediction and Management Techniques4 citations

TL;DR

This paper proposes STLinear, a minimalist traffic prediction model that replaces complex Spatial-Temporal Graph Neural Networks (STGNNs) with a fully localized architecture relying solely on linear layers. By integrating node embeddings, time series decomposition, and periodicity learning, STLinear achieves state-of-the-art accuracy while reducing computational complexity by over 95% in MACs and memory usage compared to SOTA baselines like SSTBAN.

ABSTRACT

Traffic prediction is essential for the progression of Intelligent Transportation Systems (ITS) and the vision of smart cities. While Spatial-Temporal Graph Neural Networks (STGNNs) have shown promise in this domain by leveraging Graph Neural Networks (GNNs) integrated with either RNNs or Transformers, they present challenges such as computational complexity, gradient issues, and resource-intensiveness. This paper addresses these challenges, advocating for three main solutions: a node-embedding approach, time series decomposition, and periodicity learning. We introduce STLinear, a minimalist model architecture designed for optimized efficiency and performance. Unlike traditional STGNNs, STlinear operates fully locally, avoiding inter-node data exchanges, and relies exclusively on linear layers, drastically cutting computational demands. Our empirical studies on real-world datasets confirm STLinear's prowess, matching or exceeding the accuracy of leading STGNNs, but with significantly reduced complexity and computation overhead (more than 95% reduction in MACs per epoch compared to state-of-the-art STGNN baseline published in 2023). In summary, STLinear emerges as a potent, efficient alternative to conventional STGNNs, with profound implications for the future of ITS and smart city initiatives.

Motivation & Objective

• To address the high computational complexity and resource demands of existing STGNNs in traffic prediction.
• To reduce reliance on GNNs, RNNs, and Transformers, which suffer from gradient issues, high memory usage, and scalability limitations.
• To explore whether a simple linear architecture with strategic inductive biases can match or exceed STGNN performance.
• To enable efficient deployment in communication-constrained or real-time environments through a fully localized design.
• To demonstrate that minimal architectural complexity does not compromise predictive accuracy in spatial-temporal traffic forecasting.

Proposed method

• STLinear employs a node-embedding approach to learn unique spatial attributes per node using shared parameters, mimicking soft weight sharing.
• It applies time series decomposition to extract multi-scale temporal patterns, enhancing representation learning without complex attention or recurrence.
• It integrates periodicity learning via time-of-day and day-of-week embeddings to model daily and weekly traffic cycles.
• The model is fully localized, eliminating inter-node communication during training and inference, reducing communication overhead and enabling deployment in distributed or low-bandwidth settings.
• All temporal and spatial representations are processed through pure linear transformations, avoiding non-linearities or attention mechanisms.
• The final prediction is generated via a single linear layer applied to the concatenated embeddings and decomposed time series features.

Experimental results

Research questions

• RQ1 Can a model based solely on linear layers achieve competitive traffic prediction performance compared to complex STGNNs?
• RQ2 To what extent do node-specific embeddings and periodicity modeling improve prediction accuracy in minimalist architectures?
• RQ3 How does time series decomposition contribute to performance without using RNNs or Transformers?
• RQ4 Can a fully localized model outperform standard STGNNs in both accuracy and computational efficiency?
• RQ5 What is the relative contribution of spatial attributes, daily periodicity, and weekly periodicity to the model’s predictive power?

Key findings

• STLinear achieves state-of-the-art or superior accuracy on four real-world traffic datasets, outperforming leading STGNNs including SSTBAN and PDFormer.
• The model reduces MACs per epoch by 95.50% to 99.81% and memory usage by 49.38% to 98.01% compared to SSTBAN, with reductions up to 99.96% in training MACs.
• On PEMS04-192 and PEMS04-288, SSTBAN and PDFormer encountered out-of-memory errors, while STLinear successfully trained, highlighting its scalability.
• Ablation studies confirm that time-of-day embedding is the most critical component, with its removal causing the largest performance drop.
• The absence of spatial embeddings negatively impacts performance more than missing weekly periodicity, but less than losing daily periodicity.
• STLinear maintains strong performance across all datasets, demonstrating robustness and generalization despite its minimalist design.