[Paper Review] Challenges in Moving the LEP Higgs Statistics to the LHC

K. Cranmer, B. Mellado+2 more|arXiv (Cornell University)|Dec 8, 2003

Particle physics theoretical and experimental studies6 references3 citations

TL;DR

This paper addresses the challenges of adapting LEP-era statistical methods—particularly confidence level and significance calculations using log-likelihood ratios and Fast Fourier Transform (FFT) techniques—for the high-luminosity, high-significance environment of the LHC. It introduces a solution using arbitrary precision arithmetic to overcome numerical noise limitations in FFT-based computations, enabling accurate significance estimation beyond 8σ, and advocates for power as a more meaningful alternative to significance in high-sensitivity searches.

ABSTRACT

We examine computational, conceptual, and philosophical issues in moving the statistical techniques used in the LEP Higgs working group to the LHC.

Motivation & Objective

• To address the computational and numerical challenges in transferring LEP Higgs statistical techniques to the LHC's high-significance, high-event-rate environment.
• To resolve the numerical instability in FFT-based confidence level calculations that limits significance estimation to approximately 8σ due to double-precision floating-point round-off errors.
• To propose a robust solution using arbitrary precision arithmetic to extend accurate significance calculations beyond the 8σ threshold.
• To re-evaluate the relevance of high significance values (e.g., 20σ) in discovery claims and advocate for statistical power as a more meaningful metric for discovery potential.
• To examine the philosophical and methodological divide between frequentist and Bayesian approaches to systematic uncertainties in LHC Higgs searches.

Proposed method

• Adapts the LEP statistical framework based on the log-likelihood ratio test statistic and Fast Fourier Transform (FFT) for convolution of single-event distributions to compute background-only and signal-plus-background log-likelihood ratio distributions.
• Uses FFT to efficiently compute the convolution of single-event log-likelihood ratio distributions, transforming convolution into multiplication in the frequency domain via the Fourier transform.
• Applies the Cousins-Highland method to incorporate systematic uncertainties into the significance calculation, treating them as nuisance parameters.
• Introduces arbitrary precision floating-point arithmetic to overcome numerical noise in the extreme tails of the background-only distribution, which otherwise limits significance estimation to ~8σ in double precision.
• Proposes the discovery luminosity $ L^*(m_H) $ as the integrated luminosity required for median significance to reach 5σ, acknowledging it reflects a 50% discovery probability.
• Recommends using statistical power (1 - β) as a more meaningful summary of discovery potential than high significance values, especially when significance exceeds 5σ.

Experimental results

Research questions

• RQ1 Why do standard FFT-based significance calculations fail to accurately estimate confidence levels beyond ~8σ in the LHC Higgs search context?
• RQ2 How can numerical noise in double-precision FFT computations be mitigated to enable accurate significance estimation in high-luminosity LHC searches?
• RQ3 What are the limitations of using high significance (e.g., 20σ) as a summary of discovery potential in the LHC Higgs search?
• RQ4 How does the concept of statistical power (1 - β) provide a more meaningful alternative to significance in evaluating discovery potential?
• RQ5 What are the implications of the philosophical divide between frequentist and Bayesian approaches to systematic uncertainties in LHC Higgs analyses?

Key findings

• Numerical noise in double-precision FFT computations limits accurate significance estimation to approximately 8σ, due to round-off errors on the order of 10⁻¹⁷.
• The use of arbitrary precision arithmetic resolves the numerical instability, enabling reliable computation of confidence levels and significance values beyond the 8σ threshold.
• The discovery luminosity $ L^*(m_H) $, defined as the luminosity required for median significance to reach 5σ, corresponds to a 50% chance of observing a 5σ effect under the signal-plus-background hypothesis.
• At high significance levels (e.g., 20σ), the probability of claiming discovery is not significantly higher than at 8σ, making power a more informative metric than significance.
• For a signal-plus-background hypothesis with 180 expected events and a background of 100, the statistical power is approximately 98%, demonstrating that power is a more relevant measure than significance for discovery potential.
• The paper concludes that the migration of LEP statistical tools to the LHC is non-trivial due to computational, numerical, and philosophical challenges, particularly regarding systematic uncertainties and the interpretation of high significance.