[Paper Review] Inter-Series Attention Model for COVID-19 Forecasting

Xiaoyong Jin, Yu-Xiang Wang+1 more|arXiv (Cornell University)|Oct 25, 2020

COVID-19 epidemiological studies31 references5 citations

TL;DR

This paper proposes ACTS, a data-driven time series forecasting model for COVID-19 that leverages inter-series attention to compare patterns across multiple geographic regions, significantly improving forecasting accuracy over existing methods. By combining attention mechanisms with detrending and normalization, ACTS achieves state-of-the-art performance on confirmed cases, hospitalizations, and deaths across 13 out of 18 test cases, outperforming CDC-highlighted forecasters and deep learning baselines.

ABSTRACT

COVID-19 pandemic has an unprecedented impact all over the world since early 2020. During this public health crisis, reliable forecasting of the disease becomes critical for resource allocation and administrative planning. The results from compartmental models such as SIR and SEIR are popularly referred by CDC and news media. With more and more COVID-19 data becoming available, we examine the following question: Can a direct data-driven approach without modeling the disease spreading dynamics outperform the well referred compartmental models and their variants? In this paper, we show the possibility. It is observed that as COVID-19 spreads at different speed and scale in different geographic regions, it is highly likely that similar progression patterns are shared among these regions within different time periods. This intuition lead us to develop a new neural forecasting model, called Attention Crossing Time Series ( extbf{ACTS}), that makes forecasts via comparing patterns across time series obtained from multiple regions. The attention mechanism originally developed for natural language processing can be leveraged and generalized to materialize this idea. Among 13 out of 18 testings including forecasting newly confirmed cases, hospitalizations and deaths, extbf{ACTS} outperforms all the leading COVID-19 forecasters highlighted by CDC.

Motivation & Objective

• To address the challenge of accurate, real-time forecasting of COVID-19 incidence across diverse geographic regions with limited and variable data.
• To investigate whether a purely data-driven approach without explicit disease dynamics modeling can outperform established compartmental models like SIR and SEIR.
• To develop a neural forecasting framework that captures cross-regional similarities in disease progression patterns to improve prediction accuracy.
• To evaluate the effectiveness of attention mechanisms in comparing time series across different regions for epidemic forecasting.
• To assess the contribution of detrending and normalization components in enhancing forecasting performance on sparse, non-stationary time series.

Proposed method

• The model uses an inter-series attention mechanism to compare the target region’s recent time series with historical segments from all other regions to identify similar progression patterns.
• It applies a query-key-value attention mechanism where queries are derived from the target region’s recent data, and keys are extracted from reference regions’ time series.
• A detrending module is introduced to model long-term trends that attention alone cannot capture, improving robustness to non-stationary data.
• Segment embeddings are normalized to stabilize training and improve attention alignment across regions with different scales.
• The model is trained end-to-end using only incidence data (confirmed cases, hospitalizations, deaths) and minimal auxiliary features, without relying on epidemiological assumptions.
• The attention mechanism dynamically selects relevant historical time windows from other regions based on similarity, enabling transfer of predictive patterns across geographies.

Experimental results

Research questions

• RQ1 Can a purely data-driven, attention-based model outperform established compartmental models like SIR and SEIR in forecasting COVID-19 incidence?
• RQ2 To what extent can inter-series similarity in disease progression patterns across regions improve forecasting accuracy?
• RQ3 How effective is the attention mechanism in identifying relevant historical patterns from other regions when forecasting a specific region’s future incidence?
• RQ4 What is the contribution of detrending and normalization components in improving performance on sparse and non-stationary time series?
• RQ5 Does the model’s performance improve with increasing data availability, and how does it compare to deep learning baselines like ConvTrans and TFT?

Key findings

• ACTS outperformed all leading CDC-highlighted forecasters in 13 out of 18 forecasting tasks, achieving average improvements of 9%, 5%, and 4% over the best baseline for confirmed cases, hospitalizations, and deaths, respectively.
• The model achieved the lowest WAPE (Weighted Absolute Percentage Error) across all tested dates, with performance variance of ±0.02 over five random initializations, indicating robustness.
• Inter-series attention contributed most significantly to performance gains, proving that cross-regional pattern matching enhances forecasting accuracy beyond local autoregressive modeling.
• The detrending module provided measurable improvement, suggesting that modeling long-term trends is essential when attention alone fails to capture macroscopic dynamics.
• The model trained in under 5 minutes and performed inference in seconds, demonstrating efficiency despite its complexity.
• Clustering based on attention queries revealed meaningful geographic groupings—e.g., California, Texas, and Florida were grouped together due to similar death toll trajectories—validating the model’s ability to learn biologically plausible regional similarities.