[Paper Review] Black-box Importance Sampling

Qiang Liu, Jason D. Lee|arXiv (Cornell University)|Oct 17, 2016

Statistical Methods and Inference3 references3 citations

TL;DR

This paper proposes black-box importance sampling, a method that computes importance weights for samples from any unknown or complex proposal distribution without requiring explicit knowledge of the proposal density. By formulating weight estimation as a convex quadratic optimization using kernelized Stein discrepancy, the method achieves improved estimation accuracy over standard importance sampling—even when using unnormalized, complex proposals—enabling robust inference in high-dimensional and big-data settings.

ABSTRACT

Importance sampling is widely used in machine learning and statistics, but its power is limited by the restriction of using simple proposals for which the importance weights can be tractably calculated. We address this problem by studying black-box importance sampling methods that calculate importance weights for samples generated from any unknown proposal or black-box mechanism. Our method allows us to use better and richer proposals to solve difficult problems, and (somewhat counter-intuitively) also has the additional benefit of improving the estimation accuracy beyond typical importance sampling. Both theoretical and empirical analyses are provided.

Motivation & Objective

• To overcome the limitation of traditional importance sampling, which requires tractable proposal densities and struggles with complex, high-dimensional targets.
• To enable the use of rich, complex proposals—such as those from un-converged MCMC or variational inference—by bypassing the need to compute proposal densities.
• To improve estimation accuracy despite using less information (no proposal density), challenging conventional wisdom in Monte Carlo methods.
• To provide a stable, scalable, and widely applicable method for variance reduction in unnormalized posterior estimation and approximate inference.

Proposed method

• The method formulates importance weight estimation as a convex quadratic optimization problem to minimize kernelized Stein discrepancy between the weighted empirical measure and the target distribution.
• It uses a reproducing kernel Hilbert space (RKHS) to define a discrepancy measure that depends only on the target density and the samples, not on the proposal density.
• Weights are constrained to the probability simplex (non-negative and sum to one), ensuring stability and interpretability compared to methods allowing negative or unnormalized weights.
• The approach is applicable to unnormalized target distributions, common in Bayesian statistics and machine learning, without requiring normalization constants.
• The method is implemented via a constrained optimization that can be solved efficiently using standard quadratic programming solvers.
• It leverages recent advances in Stein discrepancy to enable black-box estimation from any sample-generating mechanism, including MCMC, variational inference, or perturb-and-MAP methods.

Experimental results

Research questions

• RQ1 Can importance sampling be made effective when the proposal distribution is unknown or intractable, such as from a complex MCMC chain?
• RQ2 Does using only sample information (without proposal density) lead to better estimation accuracy than standard IS that uses proposal density?
• RQ3 Can a black-box method based on discrepancy minimization outperform traditional IS and control variate methods in high-dimensional and non-Gaussian settings?
• RQ4 How does the method perform under model misspecification or with un-converged MCMC chains commonly found in big data applications?
• RQ5 What is the impact of weight constraints (e.g., non-negativity) on the stability and accuracy of the resulting estimators?

Key findings

• The proposed black-box importance sampling method achieves lower mean squared error (MSE) than standard IS and control variate methods in high-dimensional and non-Gaussian target distributions.
• In the standard Gaussian case, the method performs comparably to the best baseline, with normalized Control Func slightly outperforming in low dimensions.
• The method significantly improves estimation accuracy over uniform weighting in both simulated and real-world Bayesian probit models, especially when MCMC chains are short or non-convergent.
• The unnormalized versions of Control Func and KDE perform poorly in practice, while their normalized variants show improved stability, highlighting the importance of proper weight normalization.
• Enforcing non-negative weights ($w_i o 0$) yields better performance in small-sample regimes ($n < 1000$), suggesting it is a robust and practical default choice.
• On the Covtype dataset with 54 features, Stein and normalized Control Func outperform uniform weighting and unnormalized alternatives, demonstrating scalability and robustness in real-world settings.