[论文解读] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows

George Papamakarios, David C. Sterratt+1 more|arXiv (Cornell University)|May 18, 2018

Gaussian Processes and Bayesian Inference参考文献 81被引用 68

一句话总结

SNL 训练一个条件自回归流以学习模拟器似然，并使用顺序轮次来引导仿真，在显著降低仿真成本的同时实现与先前神经密度方法相比具有准确性的贝叶斯推断。

ABSTRACT

We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior density. A sequential training procedure guides simulations and reduces simulation cost by orders of magnitude. We show that SNL is more robust, more accurate and requires less tuning than related neural-based methods, and we discuss diagnostics for assessing calibration, convergence and goodness-of-fit.

研究动机与目标

• Motivate Bayesian inference for simulator models with intractable likelihoods.
• Propose a general neural density estimation framework to model p(x|θ) for simulators.
• Introduce a sequential training procedure to focus simulations in high-posterior regions.
• Demonstrate robustness, calibration, and reduced simulation cost compared to existing neural methods.

提出的方法

• Model p(x|θ) with a conditional Masked Autoregressive Flow (MAF) to approximate the likelihood.
• Train qφ(x|θ) on simulated data (θ, x) from a proposal distribution ▲p(θ) to learn the likelihood in regions of interest.
• Use a sequential, round-based scheme where each round updates the posterior approximation to focus simulations where they matter, and re-train on all accumulated data.
• Employ Markov Chain Monte Carlo (Slice Sampling) to sample from the approximate posterior p̂(θ|x_o) ∝ qφ(x_o|θ) p(θ).
• Allow the proposal to concentrate in high posterior density regions without biasing the learned likelihood asymptotically, since learning is of the likelihood not the posterior.

实验结果

研究问题

• RQ1 Can Sequential Neural Likelihood (SNL) achieve accurate posterior inference with fewer simulations than existing likelihood-free methods?
• RQ2 Does learning the likelihood with an autoregressive flow and a sequential training regime provide robust calibration and good goodness-of-fit diagnostics?
• RQ3 How does SNL compare to SNPE and other likelihood-free baselines in terms of posterior accuracy and simulation efficiency?
• RQ4 What practical diagnostics can assess convergence, calibration, and goodness-of-fit for SNL?

主要发现

• SNL achieves better trade-offs between posterior accuracy and simulation cost than competing neural-density methods across several models.
• SNL is described as robust, well-calibrated, and with reduced reliance on tuning compared to related methods.
• Diagnostics show no gross calibration issues and provide convergence and goodness-of-fit checks.
• Empirical results indicate substantial simulation savings while maintaining accurate posterior estimates.
• SNL concentrates simulations in regions of high posterior density as rounds progress, accelerating learning.