[Paper Review] Event Generation and Density Estimation with Surjective Normalizing Flows

Rob Verheyen|arXiv (Cornell University)|May 3, 2022

Particle physics theoretical and experimental studies61 references46 citations

TL;DR

This paper introduces surjective and stochastic normalizing flow layers to enhance normalizing flows for particle physics event generation and density estimation. By incorporating permutation invariance, discrete feature modeling via variational dequantization and argmax surjection, and handling varying event dimensionality with a surjective transform, the method enables exact likelihood evaluation while modeling complex, real-world event features. The approach achieves state-of-the-art performance in anomaly detection on the Dark Machines challenge, particularly on unseen, complex signals.

ABSTRACT

Normalizing flows are a class of generative models that enable exact likelihood evaluation. While these models have already found various applications in particle physics, normalizing flows are not flexible enough to model many of the peripheral features of collision events. Using the framework of Nielsen et al. (2020), we introduce several surjective and stochastic transform layers to a baseline normalizing flow to improve modelling of permutation symmetry, varying dimensionality and discrete features, which are all commonly encountered in particle physics events. We assess their efficacy in the context of the generation of a matrix element-level process, and in the context of anomaly detection in detector-level LHC events.

Motivation & Objective

• To address the inflexibility of standard normalizing flows in modeling key features of LHC collision events, such as permutation symmetry, discrete features, and varying event dimensionality.
• To extend normalizing flows with surjective and stochastic transforms to maintain exact likelihood evaluation while improving modeling capacity.
• To evaluate the proposed framework on matrix element-level event generation and detector-level anomaly detection in high-energy physics.
• To compare the performance of different surjective transform strategies—stochastic permutation, sort surjection, variational dequantization, argmax surjection, and factorized models—on real-world particle physics tasks.

Proposed method

• Introduces surjective and stochastic transform layers to normalizing flows, allowing non-bijective, non-invertible transformations while preserving tractable likelihood evaluation.
• Employs a variational inference framework to model latent variables, combining the expressivity of VAEs with the exact likelihood of normalizing flows.
• Uses stochastic permutation layers to enforce permutation invariance in multi-object events, with alternatives like sort surjection for improved training stability.
• Applies variational dequantization and argmax surjection to model discrete features such as particle types or quantum numbers.
• Develops a surjective transform with vanishing bound looseness to handle events with varying numbers of objects.
• Employs a factorized model for discrete features, enabling exact likelihood computation and outperforming mixture-based alternatives.

Experimental results

Research questions

• RQ1 Can surjective and stochastic transforms be effectively integrated into normalizing flows to model permutation-invariant, discrete, and variable-dimensional particle physics events while preserving exact likelihood evaluation?
• RQ2 How do different strategies for modeling discrete features—such as variational dequantization, argmax surjection, and factorized models—affect performance in event generation and density estimation?
• RQ3 Does the use of stochastic permutation layers improve generalization in low-statistics regimes compared to deterministic sort surjection?
• RQ4 How does the proposed framework compare to existing state-of-the-art models in anomaly detection on complex, real-world LHC datasets?
• RQ5 In what scenarios does exact likelihood evaluation via factorized models provide a performance advantage over approximate likelihoods in mixture-based approaches?

Key findings

• Stochastic permutation layers outperform sort surjection in low-statistics channels (1, 2a, 2b), while sort surjection performs better in high-statistics channel 3, indicating a trade-off between generalization and data efficiency.
• The factorized model for discrete features achieves superior performance compared to mixture models, which suffer from high dimensionality and poor data efficiency due to proliferation of categories.
• Models using the proposed surjective transform for varying dimensionality achieve exact likelihood evaluation with vanishing bound looseness, enabling robust density estimation across events with different numbers of objects.
• On the secret dataset of the Dark Machines Anomaly Score Challenge, the proposed models outperform the best-performing model from [21], achieving the best results in channels 2a and 3.
• The model with stochastic permutation and dequantization surpasses the [21] flow model in channel 3, demonstrating the advantage of exact likelihood and flexible modeling in high-data regimes.
• The framework shows strong generalization to unphysical and complex signals in the secret dataset, indicating robustness to distributional shift and potential for anomaly detection in real-world scenarios.