[Paper Review] Parameter Estimation in Hidden Markov Models with Intractable Likelihoods Using Sequential Monte Carlo

Sinan Yıldırım, Sumeetpal S. Singh+2 more|arXiv (Cornell University)|Nov 17, 2013

Markov Chains and Monte Carlo Methods24 references4 citations

TL;DR

This paper proposes sequential Monte Carlo (SMC)-based maximum likelihood estimation (MLE) for hidden Markov models (HMMs) with intractable observation densities, using approximate Bayesian computation (ABC) principles to bypass likelihood evaluation. By directly estimating the score function via SMC and avoiding finite-difference approximations, the method achieves asymptotically unbiased online and offline parameter estimation, outperforming existing gradient-based methods in bias and variance control.

ABSTRACT

We propose sequential Monte Carlo based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the alpha-stable distribution, g-and-k distribution, and the stochastic volatility model with alpha-stable returns, using both real and synthetic data.

Motivation & Objective

• To address the challenge of maximum likelihood estimation in HMMs where the observation density is intractable but simulatable.
• To develop gradient-based MLE algorithms that do not require explicit evaluation of the intractable observation density.
• To provide both offline and online parameter estimation procedures using SMC and ABC-inspired techniques.
• To reduce bias and variance in parameter estimates compared to finite-difference methods, especially in long data sequences.
• To demonstrate the method’s consistency and practical performance on models with intractable densities, including α-stable, g-and-k, and stochastic volatility models.

Proposed method

• The method uses sequential Monte Carlo (SMC) to approximate the score function (gradient of the log-likelihood) directly, bypassing the need to evaluate the intractable observation density $ g_{ heta}(y|x) $.
• It leverages ABC principles by simulating from the intractable $ g_{ heta}(y|x) $ and using these simulations to construct a pseudo-likelihood for gradient estimation.
• The score function is estimated via SMC with a kernel-based approximation of the ABC tolerance, enabling consistent estimation of the gradient of the log-likelihood.
• The algorithm employs a path-space SMC scheme to propagate particles and weights, with resampling and propagation steps adapted to the intractable observation model.
• For online estimation, the method updates parameter estimates recursively using SMC-based score approximations, ensuring asymptotic unbiasedness as particle count increases.
• The approach avoids finite-difference approximations, which are known to introduce bias and high variance in gradient estimation, especially in long sequences.

Experimental results

Research questions

• RQ1 Can SMC-based MLE be effectively applied to HMMs with intractable observation densities when the likelihood cannot be evaluated directly?
• RQ2 How can the gradient of the log-likelihood be estimated without explicit knowledge of the observation density $ g_{ heta}(y|x) $, yet still yield consistent parameter estimates?
• RQ3 Does the proposed method outperform finite-difference-based MLE in terms of bias and variance, particularly in online and long-sequence settings?
• RQ4 Can the method maintain estimation consistency and reduce particle degeneracy issues compared to standard particle filters in long data streams?
• RQ5 How does the performance of the SMC-ABC MLE compare to existing Bayesian or EM-based methods in terms of convergence and accuracy for intractable HMMs?

Key findings

• The proposed SMC-based MLE algorithm achieves asymptotic unbiasedness in online estimation, unlike the finite-difference method of Ehrlich et al. (2013), which exhibits persistent bias even with exact likelihood evaluation.
• The variance of the parameter estimates decreases with increasing data length under the proposed method, whereas Ehrlich et al.’s method shows no such reduction in variance.
• The method successfully estimates parameters in three intractable models: α-stable distribution, g-and-k distribution, and stochastic volatility with α-stable returns, using both synthetic and real data.
• The SMC-based score estimation avoids the need for finite-difference approximations, reducing estimation error and improving convergence stability.
• Numerical results confirm that the method is practically consistent and outperforms existing gradient-based MLE techniques in terms of bias and variance control.
• The approach is robust to long data sequences and maintains low variance in parameter estimates, even when particle degeneracy is a concern in standard particle filters.