[논문 리뷰] Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows

George Papamakarios, David C. Sterratt외 1명|arXiv (Cornell University)|2018. 05. 18.

Model Reduction and Neural Networks인용 수 139

한 줄 요약

SNL은 조건부 자기회귀 흐름을 훈련시켜 시뮬레이션 데이터에서 p(x|θ) (가능도)를 학습하고, 후도확률이 높은 영역에 시뮬레이션을 집중시키는 순차적 스킴을 사용하여 비용을 줄이고 기존의 신경망 방법에 비해 정밀도를 향상시킵니다.

ABSTRACT

We present Sequential Neural Likelihood (SNL), a new method for Bayesian inference in simulator models, where the likelihood is intractable but simulating data from the model is possible. SNL trains an autoregressive flow on simulated data in order to learn a model of the likelihood in the region of high posterior density. A sequential training procedure guides simulations and reduces simulation cost by orders of magnitude. We show that SNL is more robust, more accurate and requires less tuning than related neural-based methods, and we discuss diagnostics for assessing calibration, convergence and goodness-of-fit.

연구 동기 및 목표

• 역불가능한 가능도를 가진 시뮬레이터 모델에 대한 베이esian 추론의 동기를 제시합니다.
• 가능도를 근사하기 위해 매개변수에 주어진 데이터에 대한 신경 밀도 추정기를 개발합니다.
• 후도확률 밀도가 높은 영역에서 시뮬레이션을 집중시키는 순차적 학습 절차를 도입합니다.
• 기존 방법과 비교하여 견고성, 보정, 시뮬레이션 비용의 감소를 입증합니다.]
• method_untouched

제안 방법

• Use a conditional Masked Autoregressive Flow (MAF) to model q_phi(x|θ).
• Train on simulated pairs (θ, x) drawn from a proposal tilde-p(θ) and p(x|θ).
• In rounds, update the proposal to concentrate near the current posterior estimate, forming a sequential scheme.
• Optimize the log-likelihood over accumulated data to fit the likelihood model without correcting for proposal bias.
• Use MCMC (Slice Sampling) to sample from the approximate posterior p̂(θ|x_o) ∝ q_phi(x_o|θ) p(θ).
• Provide diagnostics for calibration, convergence, and goodness-of-fit.

실험 결과

연구 질문

• RQ1 Can a neural density estimator model the intractable likelihood p(x|θ) accurately in the region of high posterior density?
• RQ2 Does sequentially guiding simulations to regions of high posterior density reduce simulation cost while maintaining or improving accuracy?
• RQ3 How does Sequential Neural Likelihood (SNL) compare to SNPE-A/B, SL, SMC-ABC in terms of posterior accuracy, calibration, and computational efficiency?
• RQ4 What diagnostics can assess convergence and goodness-of-fit for likelihood-free inference with neural density models?

주요 결과

• SNL achieves better trade-offs between accuracy and simulation cost than competing neural-based methods in the reported experiments.
• SNL yields well-calibrated posteriors without requiring tuning of the proposal to match Gaussian assumptions.
• The sequential training scheme concentrates simulations in high-posterior-density regions, leading to substantial reductions in total simulations.
• Diagnostics such as simulation-based calibration and posterior-data fit support indicate robust performance and convergence of SNL.
• Across toy and real-valued models (e.g., M/G/1 queue, Lotka–Volterra, Hodgkin–Huxley), SNL outperforms alternatives like NL, SNPE-A/B, SL, and SMC-ABC in efficiency and accuracy.