[Paper Review] Likelihood-free inference with emulator networks

Jan-Matthis Lueckmann, Giacomo Bassetto+2 more|arXiv (Cornell University)|May 23, 2018

Markov Chains and Monte Carlo Methods3 references45 citations

TL;DR

The paper proposes emulator networks to perform likelihood-free inference by learning flexible synthetic likelihoods with neural emulators and active learning, enabling efficient ABC without user-defined thresholds. It demonstrates local and global emulator strategies on synthetic and neuroscience examples.

ABSTRACT

Approximate Bayesian Computation (ABC) provides methods for Bayesian inference in simulation-based stochastic models which do not permit tractable likelihoods. We present a new ABC method which uses probabilistic neural emulator networks to learn synthetic likelihoods on simulated data -- both local emulators which approximate the likelihood for specific observed data, as well as global ones which are applicable to a range of data. Simulations are chosen adaptively using an acquisition function which takes into account uncertainty about either the posterior distribution of interest, or the parameters of the emulator. Our approach does not rely on user-defined rejection thresholds or distance functions. We illustrate inference with emulator networks on synthetic examples and on a biophysical neuron model, and show that emulators allow accurate and efficient inference even on high-dimensional problems which are challenging for conventional ABC approaches.

Motivation & Objective

• Enable Bayesian inference for simulator-based models with intractable likelihoods.
• Replace hand-crafted distance thresholds in ABC with probabilistic emulators of the simulator output.
• Reduce the number of simulator evaluations via active learning and acquisition strategies.
• Demonstrate scalability to high-dimensional observations and non-Gaussian noise.
• Apply the method to synthetic benchmarks and a Hodgkin-Huxley neuron model.

Proposed method

• Train neural-network based emulators q(x|θ;φ) to approximate p(x|θ) and use it as a synthetic likelihood.
• Represent uncertainty over emulator weights with probabilistic networks or ensembles to form a posterior-predictive distribution.
• Use two acquisition rules: MaxVar for local emulator learning and MaxMI for global emulator learning to guide simulations.
• Derive approximate posterior p~(θ|xo) via Hamiltonian Monte Carlo using the emulator-based synthetic likelihood.
• For global learning, maximize information gain to select informative θ for simulations.
• Apply sequential training with an initial dataset and iterative emulator updates.

Experimental results

Research questions

• RQ1 Can emulator networks accurately approximate p(x|θ) for likelihood-free inference across different data modalities including high-dimensional outputs?
• RQ2 Do active-learning acquisition rules improve simulator efficiency compared to uniform sampling or existing ABC methods?
• RQ3 How do local versus global emulators compare in terms of accuracy and computational efficiency?
• RQ4 Can the method handle non-Gaussian and heteroscedastic observation models without predefined distance metrics?
• RQ5 Is the approach effective on real scientific models such as Hodgkin-Huxley neuronal dynamics?

Key findings

• Emulators with MaxVar acquisitions converge faster to the true posterior than uniform sampling and compete with or surpass BOLFI in low-dimensional cases.
• The method scales to high-dimensional observations by learning a global emulator for the forward model, enabling posterior estimation without summary statistics.
• Applied to Hodgkin-Huxley, the approach yields posterior distributions over biophysical parameters consistent with the observed spike counts.
• Demonstrations on a blob image-generating model show faster learning of emulator parameters with MaxMI acquisitions.
• The approach avoids user-defined rejection thresholds or distance functions and handles non-Gaussian data.