[Paper Review] Online EM Algorithm for Hidden Markov Models

Olivier Cappé|arXiv (Cornell University)|Aug 17, 2009

Bayesian Methods and Mixture Models25 references4 citations

TL;DR

This paper proposes an online Expectation-Maximization (EM) algorithm for Hidden Markov Models (HMMs) that enables recursive parameter estimation from a single pass over streaming data. By combining complete-data sufficient statistics with a recursive smoothing scheme based on auxiliary recursions, the algorithm achieves convergence to stationary points of a limiting EM mapping, with performance matching batch maximum likelihood estimation for large samples.

ABSTRACT

Online (also called "recursive" or "adaptive") estimation of fixed model parameters in hidden Markov models is a topic of much interest in times series modelling. In this work, we propose an online parameter estimation algorithm that combines two key ideas. The first one, which is deeply rooted in the Expectation-Maximization (EM) methodology consists in reparameterizing the problem using complete-data sufficient statistics. The second ingredient consists in exploiting a purely recursive form of smoothing in HMMs based on an auxiliary recursion. Although the proposed online EM algorithm resembles a classical stochastic approximation (or Robbins-Monro) algorithm, it is sufficiently different to resist conventional analysis of convergence. We thus provide limited results which identify the potential limiting points of the recursion as well as the large-sample behavior of the quantities involved in the algorithm. The performance of the proposed algorithm is numerically evaluated through simulations in the case of a noisily observed Markov chain. In this case, the algorithm reaches estimation results that are comparable to that of the maximum likelihood estimator for large sample sizes.

Motivation & Objective

• To develop an online parameter estimation algorithm for HMMs that processes data sequentially without storing past observations.
• To extend the online EM framework beyond finite-memory approximations and gradient-based methods to handle general HMMs with continuous observations.
• To establish theoretical convergence properties for the proposed recursive algorithm despite its deviation from standard stochastic approximation analysis.
• To demonstrate that the algorithm achieves estimation accuracy comparable to batch maximum likelihood estimation under large-sample conditions.

Proposed method

• The algorithm reparameterizes the HMM estimation problem using complete-data sufficient statistics to align with EM principles.
• It employs a recursive smoothing scheme based on an auxiliary recursion, inspired by Zeitouni and Dembo (1988) and Elliott et al. (1995), to compute sufficient statistics online.
• The method uses a time-varying step-size sequence $ \gamma_k = k^{-1} $ to update parameter estimates recursively, ensuring convergence under mild conditions.
• The algorithm avoids storing full observation sequences by maintaining recursive sufficient statistics derived from conditional expectations given past and present data.
• It leverages the ergodicity and mixing properties of HMMs to show almost-sure convergence of empirical averages to their theoretical counterparts.
• Theoretical analysis identifies limiting points of the recursion as stationary points of a limiting EM mapping, generalizing results from independent data settings.

Experimental results

Research questions

• RQ1 Can an online EM algorithm be designed for general HMMs that avoids finite-memory approximations and enables recursive parameter updates?
• RQ2 How does the proposed algorithm's convergence behavior relate to the batch EM algorithm under large-sample conditions?
• RQ3 What are the theoretical properties of the recursive smoothing scheme used in the algorithm, and how do they ensure stability and consistency?
• RQ4 To what extent does the online algorithm achieve estimation accuracy comparable to batch maximum likelihood estimation?

Key findings

• The proposed online EM algorithm converges almost surely to stationary points of a limiting EM mapping, which correspond to the fixed points of the batch EM algorithm under large-sample conditions.
• For a noisily observed Markov chain, the algorithm achieves estimation performance comparable to the batch maximum likelihood estimator when the sample size is large.
• The recursive smoothing scheme enables exact computation of sufficient statistics without storing past observations, making the algorithm suitable for streaming data.
• Theoretical analysis shows that the empirical average of sufficient statistics converges almost surely to its expectation under the true model, ensuring consistency.
• The algorithm's convergence is established under mild mixing and moment conditions, with the step-size sequence $ \gamma_k = k^{-1} $ ensuring asymptotic convergence.
• The limiting behavior of the algorithm is characterized by the gradient of a limiting contrast function, which vanishes at the true parameter value under regularity conditions.