[Paper Review] Creating Simple, Interpretable Anomaly Detectors for New Physics in Jet Substructure

Layne Bradshaw, Spencer Chang+1 more|arXiv (Cornell University)|Mar 2, 2022

Anomaly Detection Techniques and Applications95 references30 citations

TL;DR

This paper proposes two interpretable, high-level observable-based mimickers for convolutional autoencoder-based anomaly detectors in jet substructure, using Energy Flow Polynomials (EFPs) to distill the autoencoder's decision-making. Both strategies—learning anomaly scores directly and learning pairwise event orderings—achieve ~83% agreement with the autoencoder on background events and outperform it on seven out of eight signal models, demonstrating that interpretable detectors can match complex black-box models without sacrificing performance.

ABSTRACT

Anomaly detection with convolutional autoencoders is a popular method to search for new physics in a model-agnostic manner. These techniques are powerful, but they are still a "black box," since we do not know what high-level physical observables determine how anomalous an event is. To address this, we adapt a recently proposed technique by Faucett et al., which maps out the physical observables learned by a neural network classifier, to the case of anomaly detection. We propose two different strategies that use a small number of high-level observables to mimic the decisions made by the autoencoder on background events, one designed to directly learn the output of the autoencoder, and the other designed to learn the difference between the autoencoder's outputs on a pair of events. Despite the underlying differences in their approach, we find that both strategies have similar ordering performance as the autoencoder and independently use the same six high-level observables. From there, we compare the performance of these networks as anomaly detectors. We find that both strategies perform similarly to the autoencoder across a variety of signals, giving a nontrivial demonstration that learning to order background events transfers to ordering a variety of signal events.

Motivation & Objective

• To address the 'black box' nature of deep learning anomaly detectors in high-energy physics by making their decisions interpretable.
• To develop model-agnostic, interpretable anomaly detection methods that rely on a small set of high-level observables rather than raw jet images.
• To test whether decision ordering on background events transfers to signal events, enabling practical, validated anomaly detection in experimental analyses.
• To demonstrate that simple, human-readable models can match or exceed the performance of complex autoencoders in anomaly detection tasks.

Proposed method

• Uses a convolutional autoencoder as the target anomaly detector, trained only on QCD background jet images to learn a latent representation.
• Applies a knowledge distillation-like approach to train two types of interpretable mimicker networks using high-level observables from the Energy Flow Polynomial (EFP) basis.
• The High-Level Network learns to predict the autoencoder's anomaly score directly using a small set of EFPs as input.
• The Paired Neural Network learns to order two events by predicting which is less anomalous according to the autoencoder, using a different (but overlapping) set of EFPs.
• Employs an iterative selection procedure to identify the most relevant EFPs that best reproduce the autoencoder's background decision ordering.
• Trains both mimicker networks in an unsupervised manner on background events only, ensuring generalization to unseen signal models.

Experimental results

Research questions

• RQ1 Can we distill the decision-making of a black-box autoencoder anomaly detector into a small set of interpretable high-level observables?
• RQ2 Do the mimicker networks trained on background events preserve the same decision ordering as the autoencoder across diverse signal models?
• RQ3 Which high-level observables are most critical for capturing the anomaly detection behavior of the autoencoder?
• RQ4 Can interpretable, simple models outperform the original autoencoder in detecting new physics signals?
• RQ5 Is the performance of the autoencoder on background events predictive of its performance on signal events when distilled into a transparent model?

Key findings

• Both mimicker strategies achieve ~83% agreement with the autoencoder in ordering background events, indicating strong fidelity in decision-making.
• The High-Level Network and Paired Neural Network independently identify the same six key EFPs as most informative for anomaly detection.
• The mimicker networks outperform the original autoencoder in detecting seven out of eight tested new physics signals, demonstrating transferability of background decision ordering.
• The use of only six EFPs enables high-performance anomaly detection while drastically reducing model complexity and improving interpretability.
• The results validate that the autoencoder's latent space captures physically meaningful features, such as jet substructure complexity, that can be reconstructed via simple, human-readable observables.
• The study confirms that unsupervised anomaly detection performance on background events can be effectively transferred to signal detection through interpretable models.