[Paper Review] Healthcare serial killer or coincidence? Statistical issues in investigation of suspected medical misconduct

P. J. Green, R. D. Gill+3 more|arXiv (Cornell University)|Oct 3, 2022

Medical Malpractice and Liability Issues4 citations

TL;DR

This paper provides statistical guidance for investigating suspected healthcare serial killings, emphasizing the need to distinguish rare but plausible coincidences from criminal acts. It advocates for minimizing investigative bias through expert panels, blinding, and rigorous statistical evaluation to prevent wrongful convictions based on flawed data interpretation.

ABSTRACT

Justice systems are sometimes called upon to evaluate cases in which health-care professionals are suspected of killing their patients illegally. These cases are difficult to evaluate because they involve at least two levels of uncertainty. Commonly in a murder case it is clear that a homicide has occurred, and investigators must resolve uncertainty about who is responsible. In the cases we examine here there is also uncertainty about whether homicide has occurred. Investigators need to consider whether the deaths that prompted the investigation could plausibly have occurred for reasons other than homicide, in addition to considering whether, if homicide was indeed the cause, the person under suspicion is responsible. In this report, prepared under the auspices of the Royal Statistical Society, we provide advice and guidance on the investigation and evaluation of such cases. Our work was prompted by concerns about the statistical challenges such cases pose for the legal system.

Motivation & Objective

• Address the statistical challenges in investigating suspected medical serial killings, where both the occurrence of homicide and the identity of the perpetrator are uncertain.
• Highlight the risk of misinterpreting rare event clusters as evidence of criminal behavior when they may result from chance or other factors.
• Reduce investigative bias that can distort statistical evidence, especially in cases where the suspect is also involved in data collection or analysis.
• Provide actionable recommendations for investigators, legal professionals, and statisticians to improve the reliability of statistical evidence in medical misconduct cases.
• Promote interdisciplinary collaboration between legal and statistical communities to ensure sound evaluation of statistical findings in court.

Proposed method

• Use of statistical significance testing with caution, emphasizing that low p-values do not prove causation or criminal intent.
• Application of Bayes’ rule to assess the probability of alternative theories (e.g., coincidence vs. murder) given observed data.
• Implementation of blinding in expert analysis (e.g., pathologists, toxicologists) to prevent unconscious bias from influencing results.
• Development of structured investigative procedures involving independent, multidisciplinary expert panels to assess all potential causal factors.
• Use of sensitivity and specificity analysis to evaluate the reliability of diagnostic and forensic evidence.
• Incorporation of cumulative bias modeling through numerical examples to illustrate how small biases can distort conclusions over time.

Experimental results

Research questions

• RQ1 How can statistical evidence be reliably evaluated when both the occurrence of homicide and the identity of the perpetrator are uncertain?
• RQ2 To what extent can seemingly improbable clusters of patient deaths be explained by chance or other non-criminal factors?
• RQ3 How do investigative biases—conscious or unconscious—affect the interpretation of statistical data in medical misconduct cases?
• RQ4 What methodological safeguards can be implemented to ensure statistical evidence is not distorted during data collection and analysis?
• RQ5 How can legal professionals and statisticians collaborate effectively to interpret and present statistical findings in court?

Key findings

• Seemingly unlikely event clusters in patient deaths can arise by chance, especially when multiple factors such as small sample sizes and multiple testing are involved.
• The use of p-values without regard to effect size or prior plausibility can lead to fallacious conclusions about criminal responsibility.
• Investigative bias—especially when the suspect is involved in data collection—can significantly distort statistical findings and reduce reliability.
• Blinding experts to irrelevant case details reduces the risk of unconscious bias and improves the objectivity of statistical and forensic evaluations.
• Independent expert panels composed of multidisciplinary professionals are essential to ensure balanced assessment of competing theories.
• Courts should rely on independent statisticians and cognitive bias experts when evaluating evidence from poorly conducted investigations to ensure admissibility and fairness.