[Paper Review] Consistent Adaptive Multiple Importance Sampling and Controlled Diffusions

Sep Thijssen, Hilbert J. Kappen|arXiv (Cornell University)|Mar 21, 2018

Bayesian Methods and Mixture Models15 references3 citations

TL;DR

This paper proposes a computationally efficient, consistent adaptive multiple importance sampling (AMIS) method using a novel discarding-re-weighting scheme for sampling diffusion processes. By reducing re-weighting complexity from O(K²M) to O(KM), the method achieves consistency without restrictions and outperforms the balance heuristic in computational efficiency while maintaining similar accuracy in numerical experiments.

ABSTRACT

Recent progress has been made with Adaptive Multiple Importance Sampling (AMIS) methods that show improvement in effective sample size. However, consistency for the AMIS estimator has only been established in very restricted cases. Furthermore, the high computational complexity of the re-weighting in AMIS (called balance heuristic) makes it expensive for applications involving diffusion processes. In this work we consider sequential and adaptive importance sampling that is particularly suitable for diffusion processes. We propose a new discarding-re-weighting scheme that is of lower computational complexity, and we prove that the resulting AMIS is consistent. Using numerical experiments, we demonstrate that discarding-re-weighting performs very similar to the balance heuristic, but at a fraction of the computational cost.

Motivation & Objective

• To address the high computational cost of re-weighting in Adaptive Multiple Importance Sampling (AMIS), particularly for diffusion processes.
• To establish consistency of AMIS under general conditions, overcoming prior limitations in restricted cases.
• To develop a re-weighting scheme that maintains estimator accuracy while significantly reducing computational complexity.
• To enable efficient sequential sampling in path integral control and stochastic control problems via improved AMIS.

Proposed method

• Proposes a new discarding-re-weighting scheme that computes importance weights with O(KM) complexity, compared to O(K²M) for the balance heuristic.
• Introduces a sequential AMIS framework where proposal distributions are adaptively updated using feedback from previous iterations.
• Employs path integral control to construct proposal measures via controlled SDEs, with controls parametrized as u(t,x) = A g(t,x).
• Uses Euler-Maruyama or higher-order schemes to simulate sample paths from the controlled SDEs.
• Estimates optimal control parameters A_k incrementally using weighted sums of path contributions, with weights derived from importance sampling.
• Applies the AMIS estimator to compute A_k from samples across iterations, ensuring consistency under mild regularity conditions.

Experimental results

Research questions

• RQ1 Can a re-weighting scheme be designed for AMIS that maintains estimator consistency while reducing computational complexity?
• RQ2 Does discarding-re-weighting perform comparably to the balance heuristic in terms of variance and accuracy for diffusion processes?
• RQ3 Can the proposed method be applied effectively in sequential sampling for stochastic control problems involving diffusions?
• RQ4 What is the computational complexity of the new scheme, and how does it scale with the number of proposals and samples?

Key findings

• The discarding-re-weighting scheme achieves O(KM) computational complexity, a significant improvement over the O(K²M) complexity of the balance heuristic.
• The proposed discarding-AMIS estimator is proven to be consistent under general conditions, without restrictive assumptions on proposal distributions.
• Numerical experiments show that discarding-re-weighting performs nearly identically to the balance heuristic in terms of estimator accuracy and effective sample size.
• The method enables efficient sequential sampling in path integral control, with incremental computation of feedback controls A_k across iterations.
• The algorithm scales as O(MK) when using discarding-re-weighting, compared to O(MK²) with the balance heuristic, making it practical for large-scale problems.