[Paper Review] Supervised learning improves disease outbreak detection

Benedikt Zacher, Irina Czogiel|arXiv (Cornell University)|Feb 6, 2019

Data-Driven Disease SurveillanceMedicine22 references3 citations

TL;DR

This study introduces a supervised learning approach using hidden Markov models (HMMs) to improve infectious disease outbreak detection by leveraging labeled outbreak data from public health surveillance systems. It reduces the false positive rate by up to 50% compared to the state-of-the-art FarringtonFlexible algorithm, while maintaining equivalent sensitivity, thereby decreasing workload for epidemiologists.

ABSTRACT

The early detection of infectious disease outbreaks is a crucial task to protect population health. To this end, public health surveillance systems have been established to systematically collect and analyse infectious disease data. A variety of statistical tools are available, which detect potential outbreaks as abberations from an expected endemic level using these data. Here, we develop the first supervised learning approach based on hidden Markov models for disease outbreak detection, which leverages data that is routinely collected within a public health surveillance system. We evaluate our model using real Salmonella and Campylobacter data, as well as simulations. In comparison to a state-of-the-art approach, which is applied in multiple European countries including Germany, our proposed model reduces the false positive rate by up to 50% while retaining the same sensitivity. We see our supervised learning approach as a significant step to further develop machine learning applications for disease outbreak detection, which will be instrumental to improve public health surveillance systems.

Motivation & Objective

• To develop a supervised machine learning method for early detection of infectious disease outbreaks using routinely collected surveillance data.
• To reduce the false positive rate in outbreak detection while preserving high sensitivity, thus minimizing unnecessary investigations.
• To leverage labeled outbreak data—defined by legal reporting criteria—from the Robert Koch Institute's SurvNet system to train a more accurate detection model.
• To improve the efficiency and practicality of public health surveillance by reducing the burden on epidemiologists from false alarms.
• To demonstrate the feasibility and superiority of supervised HMMs over unsupervised or traditional statistical methods in real-world outbreak detection.

Proposed method

• The method employs a supervised hidden Markov model (HMM) trained on time series of weekly case counts from multiple German counties, with state labels indicating endemic or outbreak conditions.
• The HMM models the latent state (endemic or outbreak) at each time point, with emission probabilities dependent on observed case counts and covariates such as seasonality and trends.
• Model parameters, including transition probabilities and emission distributions, are estimated using maximum likelihood with a likelihood-ratio test to assess significance of outbreak effects.
• The emission distribution is modeled as a Poisson or negative binomial distribution, with the log-linear link function to relate case counts to covariates and outbreak status.
• The model incorporates time-varying covariates such as seasonal patterns and long-term trends to improve baseline estimation.
• The final detection decision is based on the posterior probability of being in an outbreak state, computed via the forward-backward algorithm.

Experimental results

Research questions

• RQ1 Can a supervised HMM trained on labeled outbreak data from public health surveillance systems improve outbreak detection performance compared to unsupervised or traditional statistical methods?
• RQ2 To what extent can supervised learning reduce the false positive rate in disease outbreak detection without compromising sensitivity?
• RQ3 How does the inclusion of real-world outbreak labels—based on legal reporting criteria—affect model performance in detecting Salmonella and Campylobacter outbreaks?
• RQ4 Does the HMM-based approach better distinguish outbreak from endemic periods than the FarringtonFlexible algorithm, especially in the presence of small or irregular outbreaks?
• RQ5 What is the impact of outbreak size and labeling completeness on the model’s ability to detect true outbreaks?

Key findings

• The proposed HMM-based method reduced the false positive rate by up to 50% compared to the FarringtonFlexible algorithm, while maintaining the same level of sensitivity on real Salmonella and Campylobacter data.
• The model achieved significantly fewer false alarms, which could reduce the workload for epidemiologists and save time and resources in public health surveillance.
• The average number of cases in outbreak weeks was substantially higher than in endemic weeks, validating the model’s ability to distinguish between the two states.
• The estimated outbreak effect parameter (exp(β₄)) showed a strong increase in case counts during outbreak weeks, confirming the model’s ability to capture outbreak signals.
• Despite high false positive reduction, sensitivity remained high, particularly for larger outbreaks, though both methods showed low sensitivity for small outbreaks (2–3 cases).
• The model’s performance was robust even when small outbreaks were underreported, as they were not well-distinguished from endemic levels and thus did not significantly affect model learning.