[Paper Review] Variational Bayesian Optimal Experimental Design

Adam Foster, Martin Jankowiak+5 more|arXiv (Cornell University)|Mar 13, 2019

Optimal Experimental Design Methods36 references36 citations

TL;DR

The paper introduces fast estimators for expected information gain (EIG) in Bayesian optimal experimental design (BOED) using amortized variational inference, improving speed and accuracy over prior methods. It demonstrates practical end-to-end applicability on multiple experiments.

ABSTRACT

Bayesian optimal experimental design (BOED) is a principled framework for making efficient use of limited experimental resources. Unfortunately, its applicability is hampered by the difficulty of obtaining accurate estimates of the expected information gain (EIG) of an experiment. To address this, we introduce several classes of fast EIG estimators by building on ideas from amortized variational inference. We show theoretically and empirically that these estimators can provide significant gains in speed and accuracy over previous approaches. We further demonstrate the practicality of our approach on a number of end-to-end experiments.

Motivation & Objective

• Motivate BOED as a principled framework for efficient use of limited experimental resources.
• Address the computational challenge of estimating the EIG in BOED.
• Develop fast, accurate estimators of EIG leveraging amortized variational inference.
• Demonstrate the practicality of the approach on end-to-end experiments.

Proposed method

• Propose several classes of fast EIG estimators inspired by amortized variational inference.
• Theoretically analyze performance improvements over previous EIG estimation methods.
• Empirically compare speed and accuracy against baselines on synthetic and real datasets.
• Demonstrate end-to-end applicability through multiple experimental scenarios.

Experimental results

Research questions

• RQ1 Can amortized variational methods provide faster and more accurate estimates of EIG for BOED compared to existing approaches?
• RQ2 How do the proposed estimators perform in terms of computational efficiency and estimation quality across different experimental settings?
• RQ3 Do the estimators enable practical, end-to-end BOED in real-world tasks?

Key findings

• Proposed estimators offer significant gains in speed over previous EIG estimation methods.
• Estimators also show improved accuracy in estimating EIG in evaluated scenarios.
• The approach enables practical end-to-end BOED on several experiments.