[Paper Review] Localized epidemic detection in networks with overwhelming noise

Eli A. Meirom, Chris Milling+4 more|arXiv (Cornell University)|Feb 6, 2014

Complex Network Analysis Techniques4 citations

TL;DR

This paper proposes a distributed, localized algorithm that detects spreading epidemics in networks using only noisy, unreliable local reports and partial network knowledge. Despite overwhelming false positives and false negatives—where false rates exceed true signals—the algorithm accurately identifies epidemics by aggregating local reporting patterns and leveraging network topology, achieving near-perfect detection even with minimal information and multiple sources.

ABSTRACT

We consider the problem of detecting an epidemic in a population where individual diagnoses are extremely noisy. The motivation for this problem is the plethora of examples (influenza strains in humans, or computer viruses in smartphones, etc.) where reliable diagnoses are scarce, but noisy data plentiful. In flu/phone-viruses, exceedingly few infected people/phones are professionally diagnosed (only a small fraction go to a doctor) but less reliable secondary signatures (e.g., people staying home, or greater-than-typical upload activity) are more readily available. These secondary data are often plagued by unreliability: many people with the flu do not stay home, and many people that stay home do not have the flu. This paper identifies the precise regime where knowledge of the contact network enables finding the needle in the haystack: we provide a distributed, efficient and robust algorithm that can correctly identify the existence of a spreading epidemic from highly unreliable local data. Our algorithm requires only local-neighbor knowledge of this graph, and in a broad array of settings that we describe, succeeds even when false negatives and false positives make up an overwhelming fraction of the data available. Our results show it succeeds in the presence of partial information about the contact network, and also when there is not a single "patient zero", but rather many (hundreds, in our examples) of initial patient-zeroes, spread across the graph.

Motivation & Objective

• To address the challenge of detecting spreading epidemics when reliable diagnostic data are scarce but noisy secondary indicators (e.g., symptom reports, device anomalies) are abundant.
• To design a distributed, computationally efficient algorithm that operates with only local knowledge of the network structure and a single snapshot of reporting data.
• To enable accurate epidemic detection even when the number of false positives and false negatives vastly exceeds the number of true infected nodes.
• To ensure robustness to incomplete or inaccurate network topology knowledge, such as misestimated node distances or neighbor sets.
• To handle scenarios with multiple epidemic sources rather than a single 'patient zero'.

Proposed method

• The algorithm uses a localized hotspot aggregation strategy, where each node evaluates the density of reporting neighbors within a local radius to detect abnormal clustering indicative of an epidemic.
• It applies a statistical hypothesis test comparing observed local reporting density to a null model of uniform random reporting across the network.
• The method relies on local neighborhood information only—each node needs knowledge of its immediate neighbors and their reporting status, not global network structure.
• It employs a ball-based aggregation: for each node, it counts the number of reporting neighbors within a radius r, and compares this to expected counts under the null model.
• The algorithm is designed to be robust to topological noise: it tolerates errors in estimated distances or neighbor sets, with performance degradation only when reporting nodes are near the epidemic frontier.
• Theoretical analysis shows the method maintains low error rates even when the fraction of false positives and false negatives approaches 1, under broad network and reporting conditions.

Experimental results

Research questions

• RQ1 Can we detect a spreading epidemic in a network when the vast majority of reported infections are false positives and false negatives?
• RQ2 How can we detect epidemics using only a single snapshot of reporting data and local network knowledge, without observing the full time evolution of the outbreak?
• RQ3 Can the algorithm remain effective when multiple epidemic sources exist across the network, rather than a single origin?
• RQ4 How robust is the detection performance when the underlying network topology is only partially or inaccurately known?
• RQ5 Does the algorithm maintain accuracy under extreme noise levels, such as when true reporting rates are far below false reporting rates?

Key findings

• The algorithm achieves extremely low error rates—even with 0.003N true reporting nodes and 0.08N false positives—demonstrating robustness in highly noisy settings.
• Error rates decrease exponentially with network size, as predicted by theory, and remain low even on large-scale real-world networks such as Facebook (63K nodes) and Enron (33,696 nodes).
• The method successfully detects epidemics when false positive and false negative rates exceed the number of true positives, including cases where the fraction of false reports approaches 1.
• Performance degrades only when reporting nodes are near the epidemic frontier and network topology is misestimated, but type I error remains unaffected by topological noise.
• The algorithm is robust to noisy network knowledge: even with a 50% chance of distance misestimation (±1 or ±2), error rates remain low, especially for larger radii and central reporting nodes.
• The algorithm scales efficiently and is suitable for distributed deployment, as shown by low computational complexity and successful application on real-world AS-topology and email networks.