[Paper Review] Approximate Bayesian Computation for Smoothing

James S. Martin, Ajay Jasra+3 more|arXiv (Cornell University)|Jun 22, 2012

Markov Chains and Monte Carlo Methods14 references4 citations

TL;DR

This paper proposes an Approximate Bayesian Computation (ABC) framework for smoothing in Hidden Markov Models (HMMs) that avoids evaluating the observation density $ g(x,y) $ by introducing auxiliary variables in the observation space. The method achieves $ \mathcal{O}(n\epsilon) $ error in approximating smoothing expectations, with theoretical control over ABC and sequential Monte Carlo (SMC) approximation errors, enabling principled inference for models with intractable likelihoods.

ABSTRACT

We consider a method for approximate inference in hidden Markov models (HMMs). The method circumvents the need to evaluate conditional densities of observations given the hidden states. It may be considered an instance of Approximate Bayesian Computation (ABC) and it involves the introduction of auxiliary variables valued in the same space as the observations. The quality of the approximation may be controlled to arbitrary precision through a parameter ε>0 . We provide theoretical results which quantify, in terms of ε, the ABC error in approximation of expectations of additive functionals with respect to the smoothing distributions. Under regularity assumptions, this error is O(nε), where n is the number of time steps over which smoothing is performed. For numerical implementation we adopt the forward-only sequential Monte Carlo (SMC) scheme of [16] and quantify the combined error from the ABC and SMC approximations. This forms some of the first quantitative results for ABC methods which jointly treat the ABC and simulation errors, with a finite number of data and simulated samples. When the HMM has unknown static parameters, we consider particle Markov chain Monte Carlo [2] (PMCMC) methods for batch statistical inference.

Motivation & Objective

• To develop a principled ABC method for HMM smoothing that avoids evaluation of the observation density $ g(x,y) $, which is often intractable or computationally expensive.
• To quantify the combined error from ABC approximation and sequential Monte Carlo (SMC) simulation, providing theoretical control over both components.
• To enable batch inference for HMMs with unknown static parameters using particle Markov chain Monte Carlo (PMCMC) methods.
• To establish theoretical bounds on the ABC error in smoothing expectations, showing $ \mathcal{O}(n\epsilon) $ convergence with respect to the ABC tolerance $ \epsilon $.
• To extend the framework to joint inference of static parameters and latent states, supporting applications in state-space models with intractable likelihoods.

Proposed method

• Introduces an auxiliary HMM with auxiliary variables $ \tilde{Y}_n $ drawn from a $ \epsilon $-neighborhood of the true observations $ y_n $, enabling likelihood-free smoothing.
• Uses a forward-only sequential Monte Carlo (SMC) scheme to approximate the ABC smoothing distribution, avoiding direct evaluation of $ g(x_n, y_n) $.
• Employs backward representation of smoothers via backward kernels $ M_{q,\hat{\eta}_{q-1}} $ and $ M_{q,\hat{\eta}_{q-1,\epsilon},\epsilon} $ to decompose the ABC error into telescoping sums.
• Applies a telescoping sum decomposition to bound the difference between true and ABC smoothers, leveraging total variation and ergodicity properties of the backward kernels.
• Uses Theorem 2 from [22] and Lemma B.1 to bound the difference in backward kernels, showing $ \|M_{k+1,\hat{\eta}_{k,\epsilon},\epsilon} - M_{k+1,\hat{\eta}_k}\|_{TV} \leq C\epsilon $ for some $ C < \infty $.
• Combines ABC and SMC error control by analyzing the total variation distance between ABC and true filters, leading to $ \mathcal{O}(n\epsilon) $ overall error in smoothing expectations.

Experimental results

Research questions

• RQ1 What is the theoretical error bound of ABC smoothing in HMMs when the observation density $ g(x,y) $ is intractable?
• RQ2 How does the ABC approximation error in smoothing expectations scale with the tolerance $ \epsilon $ and the number of time steps $ n $?
• RQ3 Can the combined error from ABC and SMC approximations be quantified in a principled way for finite data and finite samples?
• RQ4 How can ABC smoothing be extended to joint inference of static parameters and latent states in HMMs?
• RQ5 What conditions ensure that the ABC error vanishes as $ \epsilon \to 0 $, and how fast does it converge?

Key findings

• The ABC error in approximating smoothing expectations is $ \mathcal{O}(n\epsilon) $, where $ n $ is the number of time steps and $ \epsilon > 0 $ controls the approximation tolerance.
• The error bound is uniform in $ n $, $ y_{0:n} $, and $ \epsilon $, with a constant $ C < \infty $ independent of these quantities.
• The difference between ABC and true backward kernels is bounded by $ C\epsilon $, ensuring that the ABC approximation becomes exact as $ \epsilon \to 0 $.
• The combined error from ABC and SMC approximations is quantified, forming some of the first theoretical results for ABC methods with finite data and finite samples.
• The method enables likelihood-free smoothing and inference in HMMs with intractable or costly observation densities, using forward-only SMC and PMCMC.
• Theoretical results are established under regularity assumptions, including the existence of a uniform $ C < \infty $ and geometric ergodicity $ \xi \in (0,1) $, ensuring exponential decay of error contributions.