[Paper Review] Adaptive County Level COVID-19 Forecast Models: Analysis and Improvement

Stewart W. Doe, Tyler Russell Seekins+3 more|arXiv (Cornell University)|Jun 16, 2020

COVID-19 epidemiological studies19 references4 citations

TL;DR

This paper proposes CLEIR-Net, a lightweight, hierarchical recurrent neural network that forecasts county-level COVID-19 cases by learning national time patterns via an LSTM backbone and applying a time-distributed dense layer to predict individual county changes over 14 days. It outperforms the adapted TDEFSI-LONLY model, achieving a median forecast MSE of 75,707—significantly lower than TDEFSI’s 18 million—while using far fewer parameters and improving training efficiency and generalization.

ABSTRACT

Accurately forecasting county level COVID-19 confirmed cases is crucial to optimizing medical resources. Forecasting emerging outbreaks pose a particular challenge because many existing forecasting techniques learn from historical seasons trends. Recurrent neural networks (RNNs) with LSTM-based cells are a logical choice of model due to their ability to learn temporal dynamics. In this paper, we adapt the state and county level influenza model, TDEFSI-LONLY, proposed in Wang et a. [l2020] to national and county level COVID-19 data. We show that this model poorly forecasts the current pandemic. We analyze the two week ahead forecasting capabilities of the TDEFSI-LONLY model with combinations of regularization techniques. Effective training of the TDEFSI-LONLY model requires data augmentation, to overcome this challenge we utilize an SEIR model and present an inter-county mixing extension to this model to simulate sufficient training data. Further, we propose an alternate forecast model, {\it County Level Epidemiological Inference Recurrent Network} (\alg{}) that trains an LSTM backbone on national confirmed cases to learn a low dimensional time pattern and utilizes a time distributed dense layer to learn individual county confirmed case changes each day for a two weeks forecast. We show that the best, worst, and median state forecasts made using CLEIR-Net model are respectively New York, South Carolina, and Montana.

Motivation & Objective

• To address the challenge of forecasting county-level COVID-19 cases with limited data and high dimensionality.
• To improve upon the TDEFSI-LONLY model, originally designed for influenza, for use in forecasting the SARS-CoV-2 pandemic.
• To reduce overfitting and training time by designing a parameter-efficient architecture that leverages national trends to inform local forecasts.
• To enhance model generalization through data augmentation using an inter-county mixing SEIR model to simulate realistic transmission dynamics.

Proposed method

• Adapt the TDEFSI-LONLY model, originally for influenza, to forecast county-level COVID-19 confirmed cases using recurrent neural networks with LSTM cells.
• Implement data augmentation by extending a standard SEIR model to include inflow and outflow terms between counties, simulating realistic inter-county mobility during the pandemic.
• Propose CLEIR-Net, a two-branch architecture where an LSTM backbone learns low-dimensional national time patterns from national confirmed cases.
• Use a time-distributed dense layer to map the national signal to individual county-level confirmed case changes for a 14-day forecast horizon.
• Apply regularization techniques—dropout and dynamic loss terms—on the TDEFSI model to improve generalization and reduce overfitting on simulated data.
• Train and evaluate models on real county-level data from May 18–31, 2020, comparing performance using mean squared error (MSE).

Experimental results

Research questions

• RQ1 Can a modified version of the TDEFSI-LONLY model effectively forecast county-level COVID-19 cases despite its original design for influenza and lack of mobility modeling?
• RQ2 How do different regularization techniques affect the generalization and forecasting accuracy of the adapted TDEFSI model on limited real-world data?
• RQ3 Can a novel hierarchical RNN architecture like CLEIR-Net outperform the adapted TDEFSI model in terms of forecast accuracy and parameter efficiency?
• RQ4 To what extent does incorporating inter-county mixing into the SEIR model improve the realism and utility of synthetic training data for RNNs?
• RQ5 Does leveraging national-level time patterns through a shared LSTM backbone enhance local county-level forecast performance?

Key findings

• CLEIR-Net achieved a median forecast MSE of 75,707 across U.S. counties, significantly outperforming the adapted TDEFSI model, which had an MSE of 18,798,679.
• The best-performing CLEIR-Net forecast was for New York (MSE = 7,090), while the worst was for South Carolina (MSE = 188,677), indicating strong regional performance variation.
• CLEIR-Net required only 10,945 parameters for its best-performing variant, compared to 1,038,405 parameters in the adapted TDEFSI model, demonstrating a substantial reduction in model complexity.
• The TDEFSI model with regularization techniques showed improved generalization but still underperformed the naïve no-change baseline, indicating insufficient adaptability to COVID-19 dynamics.
• Dropout regularization improved national forecasting performance in TDEFSI, outperforming the naïve baseline beyond four-day horizons, but failed to yield accurate county-level predictions.
• The proposed inter-county mixing SEIR model enhanced data augmentation realism, enabling better training of RNNs despite limited real-world county-level data.