[Paper Review] Week 49 Influenza Forecast for the 2012-2013 U.S. Season

Jeffrey Shaman, Alicia Karspeck+1 more|arXiv (Cornell University)|Dec 19, 2012

Influenza Virus Research Studies1 references3 citations

TL;DR

This paper presents a retrospective and real-time forecast of the 2012-2013 U.S. influenza season using a stochastic SIRS model with and without absolute humidity (AH) forcing, initialized in Week 49 (December 9, 2012). The key contribution is a calibrated confidence system based on historical forecast accuracy and ensemble spread, showing that peak timing predictions for major U.S. cities (e.g., Chicago, Atlanta, NYC) were accurate within ±1 week, with confidence levels exceeding 50% for most cities when validated across local, regional, and national historical data.

ABSTRACT

We present results of a forecast initiated Week 49 (beginning December 9, 2012) of the 2012-2013 influenza season for municipalities in the United States. The forecast was made on December 14, 2012. Results from forecasts initiated the two previous weeks (Weeks 47 and 48) are also presented. Also results from the forecast generated with the SIRS model without AH forcing (no AH) are shown

Motivation & Objective

• To forecast the peak timing of influenza in U.S. cities during the 2012-2013 season using a stochastic SIRS model.
• To evaluate forecast skill using retrospective data from Weeks 47–49, focusing on accuracy and ensemble spread.
• To develop a confidence calibration system based on historical relationships between forecast accuracy and ensemble variance.
• To compare model performance with and without absolute humidity forcing to assess its impact on prediction reliability.
• To assess regional and municipal variability in forecast predictability across different U.S. census regions.

Proposed method

• A stochastic Susceptible-Infected-Recovered (SIRS) model was used to simulate influenza transmission dynamics in U.S. municipalities.
• The model was forced with climatological absolute humidity (AH) data and also tested without AH forcing to assess its influence on forecast skill.
• Forecasts were initialized weekly (Weeks 47–49) using observed ILI (influenza-like illness) data up to the start of each forecast week.
• Ensemble forecasts were generated with multiple model realizations to estimate uncertainty, and ensemble spread was quantified using log variance.
• Calibrated confidence levels were assigned based on historical relationships between forecast accuracy and ensemble spread at city, regional, and national levels.
• Peak week predictions were evaluated using ensemble mode estimates, with confidence levels derived from historical performance across local, regional, and national baselines.

Experimental results

Research questions

• RQ1 How accurate are SIRS model forecasts for influenza peak timing in U.S. cities when initialized in Week 49 of the 2012-2013 season?
• RQ2 What is the relationship between ensemble forecast spread and actual prediction accuracy across different U.S. cities and regions?
• RQ3 How does including absolute humidity forcing in the SIRS model affect forecast skill and confidence calibration compared to the no-AH version?
• RQ4 To what extent does forecast confidence vary between local, regional, and national historical calibration methods?
• RQ5 Why do some cities (e.g., St. Louis, NYC) show high forecast confidence while others (e.g., East North Central region) show limited skill, and how does this vary by region?

Key findings

• The SIRS model with climatological absolute humidity forcing predicted peak influenza activity in Atlanta, Chicago, and Dallas for Week 49 (December 8, 2012), with confidence levels >50% for most cities.
• New York City’s peak was forecast to occur in 2–3 weeks (Week 51–52) with 50% confidence, a significant shift from the prior week’s prediction of 4–5 weeks ahead.
• St. Louis and Miami were predicted to peak in 1–2 weeks (Week 50–51) with high confidence, and the model with no AH forcing predicted a similar peak timing, though slightly earlier by 1 week.
• The forecast skill plateaued when ensemble log variance dropped below 2.5–3.0, indicating a threshold beyond which further reduction in spread did not improve accuracy.
• Regional heterogeneity in forecast skill was observed: the West North Central region showed strong skill across all lead times, while the Mountain and East North Central regions showed limited skill, especially at longer leads.
• The no-AH model produced slightly earlier peak predictions for Los Angeles and San Francisco (1–2 weeks vs. 2–3 weeks in the AH model), but within the ±1 week margin of error, indicating consistency across model variants.