[Paper Review] On the predictability of infectious disease outbreaks

Samuel V. Scarpino, Giovanni Petri|arXiv (Cornell University)|Mar 21, 2017

Evolution and Genetic Dynamics54 references4 citations

TL;DR

This study investigates the fundamental limits to predicting infectious disease outbreaks using permutation entropy as a model-independent measure of predictability across 10 historical diseases. It identifies a universal entropy barrier beyond which accurate forecasting becomes impossible, yet for most diseases this barrier lies far beyond single-outbreak timescales, implying reliable short-to-mid-term forecasts are feasible—especially when dynamic, data-driven models account for shifting transmission dynamics and social network heterogeneity.

ABSTRACT

Infectious disease outbreaks recapitulate biology: they emerge from the multi-level interaction of hosts, pathogens, and their shared environment. As a result, predicting when, where, and how far diseases will spread requires a complex systems approach to modeling. Recent studies have demonstrated that predicting different components of outbreaks--e.g., the expected number of cases, pace and tempo of cases needing treatment, demand for prophylactic equipment, importation probability etc.--is feasible. Therefore, advancing both the science and practice of disease forecasting now requires testing for the presence of fundamental limits to outbreak prediction. To investigate the question of outbreak prediction, we study the information theoretic limits to forecasting across a broad set of infectious diseases using permutation entropy as a model independent measure of predictability. Studying the predictability of a diverse collection of historical outbreaks--including, chlamydia, dengue, gonorrhea, hepatitis A, influenza, measles, mumps, polio, and whooping cough--we identify a fundamental entropy barrier for infectious disease time series forecasting. However, we find that for most diseases this barrier to prediction is often well beyond the time scale of single outbreaks. We also find that the forecast horizon varies by disease and demonstrate that both shifting model structures and social network heterogeneity are the most likely mechanisms for the observed differences across contagions. Our results highlight the importance of moving beyond time series forecasting, by embracing dynamic modeling approaches, and suggest challenges for performing model selection across long time series. We further anticipate that our findings will contribute to the rapidly growing field of epidemiological forecasting and may relate more broadly to the predictability of complex adaptive systems.

Motivation & Objective

• To determine whether there are intrinsic, fundamental limits to predicting infectious disease outbreaks, independent of model quality or data availability.
• To assess whether the predictability of disease time series is constrained by inherent system complexity, as measured by information-theoretic entropy.
• To evaluate whether long-term forecasting accuracy degrades not due to data scarcity, but due to structural and dynamic changes in transmission patterns.
• To investigate how factors like social network heterogeneity and shifting transmission dynamics affect predictability across different diseases.
• To challenge the assumption that longer time series always improve forecast accuracy, particularly in complex, adaptive sociobiological systems.

Proposed method

• Applied permutation entropy—a model-free, information-theoretic measure of time series complexity and predictability—across 10 historical infectious disease time series (e.g., measles, influenza, dengue).
• Used 52-week sliding windows to compute permutation entropy over time, identifying periods of high and low predictability in real-world data.
• Compared observed entropy values to those from stochastic (white noise) and deterministic (noisy sine wave) reference processes to establish baseline predictability levels.
• Integrated biological and epidemiological knowledge with simulated outbreaks to explore how changes in transmission dynamics (e.g., vaccination coverage, secondary attack rates) affect predictability.
• Evaluated the impact of data length on predictability, testing whether longer time series consistently improve forecast performance.
• Employed cross-validation and model selection techniques to assess whether long time series reliably favor the correct model structure, testing for a 'no free lunch' scenario in model selection.

Experimental results

Research questions

• RQ1 Is there a fundamental, information-theoretic limit to the predictability of infectious disease outbreaks, independent of modeling or data quality?
• RQ2 How does the predictability of disease time series vary over time, and what factors drive fluctuations in predictability?
• RQ3 To what extent do changes in transmission dynamics—such as shifts in social network structure or vaccination coverage—affect forecast accuracy?
• RQ4 Does increasing the length of time series data always improve forecast performance, or can it lead to spurious predictability in complex systems?
• RQ5 Can dynamic, data-driven models overcome the entropy barriers identified in time series forecasting, and what does this imply for model selection in epidemiology?

Key findings

• A universal entropy barrier exists in infectious disease time series, representing a fundamental limit to predictability beyond which forecasting becomes statistically infeasible.
• For most diseases studied—including measles, influenza, and dengue—the entropy barrier lies well beyond the typical duration of a single outbreak, indicating that reliable short- to mid-term forecasts are generally achievable.
• Permutation entropy values in real disease data fluctuate significantly over time, with periods of high predictability (resembling deterministic signals) and low predictability (resembling white noise), suggesting that forecast accuracy is time-dependent.
• Increasing data length does not always improve predictability; in some cases, longer time series lead to lower predictability due to structural shifts in transmission dynamics, such as changes in contact networks or immunity levels.
• Model performance degradation with longer data spans suggests that the optimal model structure may vary over time and scale, supporting the need for adaptive, iteratively calibrated models.
• The findings imply a 'no free lunch' scenario in model selection for infectious disease forecasting, where no single model structure consistently outperforms others across all time windows, especially in non-stationary systems.