[Paper Review] Anomaly Detection for Physics Analysis and Less than Supervised Learning

Benjamin Nachman|arXiv (Cornell University)|Oct 27, 2020

Computational Physics and Python Applications4 citations

TL;DR

This paper introduces machine learning-based anomaly detection methods for high-energy physics (HEP) that reduce reliance on predefined signal and background models, enabling more sensitive, model-agnostic searches for new physics. It presents novel approaches like CWoLa and ANODE that learn from data directly, achieving strong limits on new particle production in dijet final states without prior signal simulations, marking a shift toward data-driven discovery in HEP.

ABSTRACT

Modern machine learning tools offer exciting possibilities to qualitatively change the paradigm for new particle searches. In particular, new methods can broaden the search program by gaining sensitivity to unforeseen scenarios by learning directly from data. There has been a significant growth in new ideas and they are just starting to be applied to experimental data. This chapter introduces these new anomaly detection methods, which range from fully supervised algorithms to unsupervised, and include weakly supervised methods.

Motivation & Objective

• To develop anomaly detection methods in high-energy physics that are less dependent on predefined signal and background models.
• To enable sensitive searches for new physics by learning directly from data, even when signal models are unknown or hard to simulate.
• To reduce model dependence in both test statistic selection and background calibration, improving robustness and broadening the scope of new physics searches.
• To demonstrate the feasibility of data-driven anomaly detection with real collider data, moving beyond simulation-only validation.

Proposed method

• Uses weakly supervised and unsupervised machine learning techniques such as CWoLa (Classification Without Labels) and ANODE (Anomaly Detection with Density Estimation) to identify deviations from expected background.
• Employs density estimation in multidimensional feature space to model background distributions without requiring full signal simulations.
• Applies likelihood ratio-based test statistics optimized via neural networks, enabling optimal sensitivity without assuming a specific signal model.
• Utilizes data-driven density estimation and sideband methods to calibrate p-values without relying on Monte Carlo simulations for background.
• Trains multiple neural networks (on the order of 10^4) to retrain the classifier for each injected signal model, enabling sensitivity scans across diverse signal hypotheses.
• Visualizes classifier outputs in 2D feature space to demonstrate automatic signal identification, enabling interpretability and validation.

Experimental results

Research questions

• RQ1 Can machine learning-based anomaly detection methods achieve sensitivity to new physics without relying on detailed signal models?
• RQ2 How can background p-values be calibrated in the absence of full Monte Carlo simulations using data-driven density estimation?
• RQ3 To what extent can weakly supervised methods like CWoLa detect new physics in complex final states such as dijets with collimated decay products?
• RQ4 How does the performance of model-agnostic anomaly detection compare to traditional, model-dependent searches in terms of discovery reach?
• RQ5 What are the computational and methodological challenges in applying these methods to real collider data?

Key findings

• The first data-driven anomaly detection search using the CWoLa method was successfully performed by ATLAS, demonstrating automatic signal identification in a dijet final state.
• The CWoLa-based analysis achieved the strongest limits on new physics production cross sections for certain signal models, outperforming traditional methods in specific scenarios.
• Despite no evidence for new particles, the study validated the feasibility of model-agnostic anomaly detection with real data, marking a milestone in HEP.
• The method required training approximately 10^4 neural networks to scan over different signal masses and strengths, highlighting the computational cost of such approaches.
• Visualizations of the classifier output confirmed that the model could automatically identify signal-like regions even without prior signal injection in the training phase.
• The LHC Olympics 2020 data challenge was established to further develop and benchmark these methods in a realistic, mixed-simulation environment.