[Paper Review] Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information

Willie Neiswanger, Ke Alexander Wang+1 more|arXiv (Cornell University)|Apr 19, 2021

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper introduces Bayesian Algorithm Execution (BAX), a framework for estimating computable properties of expensive black-box functions using a limited budget of function evaluations. It proposes InfoBAX, an iterative query selection method that maximizes mutual information between function evaluations and the algorithmic output, reducing required queries by up to 500 times compared to full algorithm execution.

ABSTRACT

In many real-world problems, we want to infer some property of an expensive black-box function $f$, given a budget of $T$ function evaluations. One example is budget constrained global optimization of $f$, for which Bayesian optimization is a popular method. Other properties of interest include local optima, level sets, integrals, or graph-structured information induced by $f$. Often, we can find an algorithm $\\mathcal{A}$ to compute the desired property, but it may require far more than $T$ queries to execute. Given such an $\\mathcal{A}$, and a prior distribution over $f$, we refer to the problem of inferring the output of $\\mathcal{A}$ using $T$ evaluations as Bayesian Algorithm Execution (BAX). To tackle this problem, we present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output. Applying this to Dijkstra's algorithm, for instance, we infer shortest paths in synthetic and real-world graphs with black-box edge costs. Using evolution strategies, we yield variants of Bayesian optimization that target local, rather than global, optima. On these problems, InfoBAX uses up to 500 times fewer queries to $f$ than required by the original algorithm. Our method is closely connected to other Bayesian optimal experimental design procedures such as entropy search methods and optimal sensor placement using Gaussian processes.

Motivation & Objective

• To address the challenge of estimating computable properties of expensive black-box functions under a strict evaluation budget.
• To unify diverse problems—such as global/local optimization, level sets, integrals, and graph properties—under a single Bayesian inference framework.
• To develop a method that selects function evaluations adaptively to maximize information about the output of a given algorithm $σ$ without modifying $σ$ to respect the budget.
• To enable accurate inference of algorithm outputs using far fewer queries than required by the original algorithm, especially in settings with noisy or costly evaluations.

Proposed method

• InfoBAX uses sequential query selection to maximize the mutual information (MI) between function evaluations and the output of a target algorithm $σ$.
• It formulates the acquisition function as the expected information gain (EIG) in the algorithm’s output $O_{σ}$ given a new evaluation at input $x$, using posterior entropy reduction as the criterion.
• The method relies on Monte Carlo sampling from the posterior GP model to simulate the algorithm $σ$ on multiple function samples, enabling estimation of MI without closed-form expressions.
• Three variants of the EIG objective are proposed: $σ_{t}^{e}(x)$ (execution path), $σ_{t}(x)$ (output), and $σ_{t}^{v}(x)$ (value-based), each capturing different aspects of algorithmic uncertainty.
• The approach is grounded in Bayesian optimal experimental design, extending entropy search and optimal sensor placement to algorithmic outputs.
• It supports noisy function evaluations by modeling $f$ with a Gaussian process and propagating uncertainty through the algorithm $σ$.

Experimental results

Research questions

• RQ1 Can we estimate the output of a computable algorithm $σ$ on a black-box function $f$ using significantly fewer than $T$ function evaluations, where $T$ is the number required by $σ$?
• RQ2 How can we design a query selection strategy that maximizes information gain about the algorithm’s output, rather than just about the function values?
• RQ3 What are effective approximations to the mutual information between function evaluations and the output of a general-purpose algorithm $σ$?
• RQ4 How does InfoBAX compare to standard baselines like random search and uncertainty sampling in terms of convergence speed and accuracy for algorithmic property estimation?
• RQ5 Can the method be applied to diverse algorithmic tasks such as top-$k$ selection, shortest path computation, and local optimization?

Key findings

• InfoBAX reduces the number of required function evaluations by up to 500 times compared to executing the full algorithm $σ$ in synthetic and real-world graph problems.
• The variant of InfoBAX using $σ_{t}^{v}(x)$, which maximizes information about the algorithm’s output values, achieved the fastest convergence to the true top-$k$ set in the top-$k$ estimation experiment.
• In the top-$k$ estimation task, InfoBAX with $σ_{t}^{v}(x)$ and $σ_{t}(x)$ achieved a Jaccard distance of less than 0.1 after 70 evaluations, outperforming UncertaintySampling and RandomSearch.
• The method successfully inferred shortest paths in graphs with black-box edge costs using Dijkstra’s algorithm, demonstrating applicability to graph-structured properties.
• InfoBAX focused its queries on the true top-$k$ elements (gold stars) and the algorithm’s execution path (light grey dots), while baselines like UncertaintySampling targeted high-uncertainty regions of the full function space.
• The approach generalizes across diverse algorithmic tasks, including local optimization via evolution strategies and level set estimation, without modifying the original algorithm $σ$.