[Paper Review] Using machine learning to auto-tune chi-squared tests for gravitational wave searches

C. McIsaac, I. W. Harry|arXiv (Cornell University)|Mar 7, 2022

Pulsars and Gravitational Waves Research56 references13 citations

TL;DR

This paper proposes a machine learning framework to auto-tune chi-squared (χ²) signal-consistency tests in gravitational wave searches, using stochastic gradient descent to optimize parameters that distinguish astrophysical signals from non-Gaussian noise glitches. The method improves search sensitivity by ~11% for intermediate-mass black hole binaries with total masses >300M⊙ by effectively down-weighting noise while preserving signal response.

ABSTRACT

The sensitivity of gravitational wave searches is reduced by the presence of non-Gaussian noise in the detector data. These non-Gaussianities often match well with the template waveforms used in matched filter searches, and require signal-consistency tests to distinguish them from astrophysical signals. However, empirically tuning these tests for maximum efficacy is time consuming and limits the complexity of these tests. In this work we demonstrate a framework to use machine-learning techniques to automatically tune signal-consistency tests. We implement a new $\chi^2$ signal-consistency test targeting the large population of noise found in searches for intermediate mass black hole binaries, training the new test using the framework set out in this paper. We find that this method effectively trains a complex model to down-weight the noise, while leaving the signal population relatively unaffected. This improves the sensitivity of the search by $\sim 11\%$ for signals with masses $> 300 M_\odot$. In the future this framework could be used to implement new tests in any of the commonly used matched-filter search algorithms, further improving the sensitivity of our searches.

Motivation & Objective

• To address the challenge of non-Gaussian noise transients (glitches) in gravitational wave detectors that mimic compact binary coalescence signals and degrade search sensitivity.
• To overcome the time-consuming and limited nature of manual tuning of signal-consistency tests in matched-filter searches.
• To develop a scalable, learnable framework for training complex χ² tests that can be integrated into existing matched-filter search pipelines.
• To improve sensitivity in searches for intermediate-mass black hole binaries, where high-mass signals are particularly vulnerable to noise contamination.
• To demonstrate that machine learning can effectively optimize signal-consistency tests without replacing the matched filter, preserving statistical rigor and network-coincidence testing.

Proposed method

• The framework uses stochastic gradient descent (SGD) to train a new χ² signal-consistency test by optimizing tunable parameters that weight different frequency bands in the matched filter response.
• The training is performed on a dataset of noise triggers from prior searches and simulated gravitational wave signals, using a loss function that maximizes separation between noise and signal populations.
• The new χ² test is constructed using orthogonal templates to the main signal template, enabling a reduced χ² distribution under Gaussian noise assumptions.
• The model is trained to assign higher χ² values to noise-like triggers while maintaining low values for true signals, effectively down-weighting noise in the reweighted SNR.
• The method is implemented within the PyCBC search framework, preserving existing network-coincidence and detection statistic workflows.
• The trained model is validated using a non-astrophysical background of time-shifted triggers to estimate false alarm rates and sensitivity gains.

Experimental results

Research questions

• RQ1 Can machine learning techniques be used to automatically tune signal-consistency tests in gravitational wave searches to improve sensitivity?
• RQ2 How effective is stochastic gradient descent in optimizing a χ² test to distinguish non-Gaussian noise from astrophysical signals?
• RQ3 To what extent does the learned χ² test improve sensitivity in searches for intermediate-mass black hole binaries with high total mass?
• RQ4 Does the learned test maintain statistical rigor and compatibility with standard matched-filter pipelines, including network coincidence tests?
• RQ5 Can this framework be generalized to other search algorithms and signal classes without compromising detection performance?

Key findings

• The machine learning–based auto-tuning framework successfully trained a complex χ² signal-consistency test that effectively down-weights non-Gaussian noise while preserving the signal response.
• The new test improved the sensitivity of the intermediate-mass black hole search by approximately 11% for signals with total masses >300M⊙.
• The sensitivity gain was most pronounced at high masses, where the signal duration is short and noise transients are more likely to mimic the signal morphology.
• The trained model maintained statistical consistency and could be integrated into existing matched-filter search pipelines without altering core detection statistics.
• The framework demonstrated robustness to unseen data, as the χ² test remains rigorous under Gaussian noise assumptions, even when applied to new glitch populations.
• The method provides a scalable and generalizable approach to signal-consistency testing that can be applied to any modelled matched-filter search algorithm.