[Paper Review] Hierarchical Neural Simulation-Based Inference Over Event Ensembles

L. Heinrich, Siddharth Mishra-Sharma+2 more|arXiv (Cornell University)|Jun 21, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes a hierarchical neural simulation-based inference framework that leverages dataset-wide learning to improve parameter estimation in models with intractable likelihoods, using deep sets and normalizing flows to jointly infer global and local parameters from event ensembles. It achieves faster, consistent inference than MCMC and enables real-time updates in streaming settings.

ABSTRACT

When analyzing real-world data it is common to work with event ensembles, which comprise sets of observations that collectively constrain the parameters of an underlying model of interest. Such models often have a hierarchical structure, where "local" parameters impact individual events and "global" parameters influence the entire dataset. We introduce practical approaches for frequentist and Bayesian dataset-wide probabilistic inference in cases where the likelihood is intractable, but simulations can be realized via a hierarchical forward model. We construct neural estimators for the likelihood(-ratio) or posterior and show that explicitly accounting for the model's hierarchical structure can lead to significantly tighter parameter constraints. We ground our discussion using case studies from the physical sciences, focusing on examples from particle physics and cosmology.

Motivation & Objective

• To address the limitation of existing simulation-based inference methods that treat individual events in isolation, ignoring the hierarchical structure of event ensembles.
• To develop a unified framework for dataset-wide inference in implicit models where likelihoods are intractable but forward simulations are available.
• To improve parameter constraint accuracy by explicitly modeling the interplay between global parameters (affecting all events) and local parameters (affecting individual events).
• To enable efficient, amortized inference that supports real-time updates as new data arrives, without retraining on the full dataset.
• To demonstrate the approach across real-world scientific domains, including particle physics and astrophysics, using both Bayesian and frequentist inference paradigms.

Proposed method

• Uses deep set architectures to model permutation-equivariant representations of event ensembles, enabling joint inference over variable-sized datasets.
• Employs normalizing flows as amortized posterior estimators for both global and local parameters, with shared embeddings conditioned on the full dataset.
• Applies a hierarchical likelihood-ratio estimator in the frequentist setting, trained via binary classification on simulated datasets with perturbed global parameters.
• Integrates set-wide and per-event encoders: the former aggregates event-level features to estimate global parameters, while the latter encodes individual event data.
• Uses a joint optimization objective combining log-likelihoods from global and local normalizing flows to train the end-to-end model.
• Supports varying dataset cardinality by conditioning the global decoder on the number of events in the ensemble, enabling generalization across different-sized datasets.

Experimental results

Research questions

• RQ1 Can dataset-wide inference that respects the hierarchical structure of event ensembles lead to tighter and more accurate parameter constraints than per-event inference?
• RQ2 How can simulation-based inference be adapted to handle variable-sized event ensembles while maintaining statistical efficiency and scalability?
• RQ3 To what extent can neural estimators for likelihood ratios or posteriors outperform traditional MCMC methods in terms of speed and accuracy when likelihoods are intractable?
• RQ4 Can the proposed framework support real-time inference in streaming data scenarios, where posterior estimates are incrementally updated as new events arrive?
• RQ5 How do the proposed methods compare to existing SBI techniques in terms of performance on real-world physical science problems like particle collider data and gravitational lensing?

Key findings

• The proposed hierarchical neural simulation-based inference framework achieves significantly faster inference than MCMC, even when the likelihood is tractable, due to amortized learning and end-to-end training.
• Explicitly modeling the hierarchical structure of global and local parameters leads to tighter posterior constraints and improved inference accuracy compared to treating events independently.
• The method supports real-time inference in streaming mode, enabling efficient posterior updates upon arrival of new data without reprocessing the entire dataset.
• In the particle physics case study, the frequentist likelihood-ratio estimator trained via deep sets achieved high statistical power with minimal calibration, outperforming standard profile likelihood approaches.
• In the astrophysics case study on strong gravitational lensing, the normalizing flow-based posterior estimator accurately recovered global parameters (α, β) and local image parameters (θ_E, Δx, Δy, q) with high fidelity, even with noisy and complex image data.
• The framework generalizes across datasets of varying cardinality, as demonstrated by training on ensembles of different sizes and achieving consistent performance on held-out test sets.