[Paper Review] Learning to Transfer for Traffic Forecasting via Multi-task Learning

Yichao Lu|arXiv (Cornell University)|Nov 27, 2021

Traffic Prediction and Management Techniques51 references4 citations

TL;DR

This paper proposes a multi-task learning framework that enables deep learning models to adapt to temporal and spatio-temporal domain shifts in traffic forecasting, using a U-Net architecture trained jointly on data from different cities and time periods. The approach achieves state-of-the-art performance in the NeurIPS 2021 Traffic4cast challenge, ranking 2nd in the extended competition with minimal computational overhead.

ABSTRACT

Deep neural networks have demonstrated superior performance in short-term traffic forecasting. However, most existing traffic forecasting systems assume that the training and testing data are drawn from the same underlying distribution, which limits their practical applicability. The NeurIPS 2021 Traffic4cast challenge is the first of its kind dedicated to benchmarking the robustness of traffic forecasting models towards domain shifts in space and time. This technical report describes our solution to this challenge. In particular, we present a multi-task learning framework for temporal and spatio-temporal domain adaptation of traffic forecasting models. Experimental results demonstrate that our multi-task learning approach achieves strong empirical performance, outperforming a number of baseline domain adaptation methods, while remaining highly efficient. The source code for this technical report is available at https://github.com/YichaoLu/Traffic4cast2021.

Motivation & Objective

• To address the challenge of domain shift in traffic forecasting, particularly due to temporal changes (e.g., pandemic-induced) and spatial shifts (new cities).
• To develop a transferable and robust forecasting model that generalizes across different cities and time periods.
• To improve model performance under distributional shifts without requiring task-specific fine-tuning per city or time period.
• To demonstrate that multi-task learning can implicitly enable domain adaptation through joint training on diverse data.

Proposed method

• A U-Net-based deep neural network is trained using multi-task learning on traffic flow data from multiple cities and time periods (2019 and 2020).
• The model is jointly optimized across all source domains (cities and years) to learn shared representations that generalize across distribution shifts.
• The training setup includes end-to-end learning with a single model for all cities and time periods, avoiding per-city model fitting.
• Early stopping is applied to prevent overfitting, especially in low-data regimes.
• The approach leverages spatio-temporal data as video-like sequences, with 2D convolutions to model spatial and temporal dependencies.
• Model ensembling is used in inference to further improve performance on the leaderboard.

Experimental results

Research questions

• RQ1 Can multi-task learning effectively enable domain adaptation in traffic forecasting under significant temporal and spatial distribution shifts?
• RQ2 How does joint training on data from multiple cities and years compare to training on individual cities or time periods?
• RQ3 Does multi-task learning improve generalization to unseen cities and post-pandemic traffic patterns compared to standard domain adaptation baselines?
• RQ4 To what extent does data from the same city and similar time periods (e.g., 2020 vs. 2020 validation) improve model performance?
• RQ5 Can a simpler model like U-Net outperform more complex architectures (e.g., Graph ResNet) when trained with multi-task learning and proper regularization?

Key findings

• The multi-task learning approach achieved 2nd place in the NeurIPS 2021 Traffic4cast extended competition (spatio-temporal domain adaptation), with a leaderboard score of 59.586, just 0.05% behind the top team.
• The method outperformed a Graph ResNet baseline, which previously achieved SOTA performance, demonstrating that architectural simplicity combined with multi-task learning can be more effective.
• Training on data from the same city (e.g., Bangkok 2019) was more beneficial than collecting data from multiple cities, highlighting the importance of distributional similarity.
• Combining 2019 and 2020 data from a single city (Barcelona) outperformed using 2019 or 2020 data from three cities, indicating that multi-task learning enhances temporal domain adaptation.
• The model achieved strong performance with minimal computational cost, as it required training only one model for all cities and time periods.
• The results suggest that multi-task learning implicitly enables domain adaptation by encouraging the model to learn invariant features across domains.