[Paper Review] Hidden Markov Model Applications in Change-Point Analysis

The Minh Luong, Vittorio Perduca+1 more|arXiv (Cornell University)|Dec 8, 2012

Gene expression and cancer classification49 references17 citations

TL;DR

This paper presents a unified Hidden Markov Model (HMM) framework for change-point analysis, enabling efficient inference on segment boundaries and hidden states using forward-backward algorithms and Bayesian networks. It demonstrates that HMMs provide a flexible, computationally tractable approach for detecting change-points in heterogeneous sequences across diverse applications, with extensions supporting joint segmentation and uncertainty quantification.

ABSTRACT

The detection of change-points in heterogeneous sequences is a statistical challenge with many applications in fields such as finance, signal analysis and biology. A wide variety of literature exists for finding an ideal set of change-points for characterizing the data. In this tutorial we elaborate on the Hidden Markov Model (HMM) and present two different frameworks for applying HMM to change-point models. Then we provide a summary of two procedures for inference in change-point analysis, which are particular cases of the forward-backward algorithm for HMMs, and discuss common implementation problems. Lastly, we provide two examples of the HMM methods on available data sets and we shortly discuss about the applications to current genomics studies. The R code used in the examples is provided in the appendix.

Motivation & Objective

• To unify segment-based and level-based change-point models under a common HMM formalism for improved inference and modeling flexibility.
• To enable efficient computation of posterior state distributions and change-point locations using linear-time forward-backward recursions.
• To support model selection and uncertainty quantification through Bayesian criteria like BIC and ICL within the HMM framework.
• To extend HMMs to handle joint segmentation across multiple samples and complex dependencies between segments and levels.
• To provide a scalable, probabilistic foundation for change-point detection applicable in genomics, finance, and signal processing.

Proposed method

• Formalizes change-point detection as a hidden Markov model where observed data are emissions and segment indices are hidden states.
• Uses the forward-backward algorithm to compute posterior marginal state distributions P(S_i | X_{1:n}) in O(nK) time for K segments.
• Applies the expectation-maximization (EM) algorithm and MCMC methods (e.g., Gibbs sampling, reversible jump MCMC) for parameter estimation.
• Integrates Bayesian network principles to generalize HMMs, enabling recursive belief propagation and junction tree algorithms for exact inference.
• Extends the model to include joint segmentation across multiple samples using a shared segmentation variable S_i.
• Incorporates evidence constraints via conditional probability propagation to handle partial observations or prior knowledge on states.

Experimental results

Research questions

• RQ1 How can segment-based and level-based change-point models be unified under a single HMM framework?
• RQ2 What are the computational and statistical advantages of using HMMs for change-point detection compared to classical segmentation methods?
• RQ3 How can posterior uncertainty in change-point locations be quantified using HMM inference algorithms?
• RQ4 In what ways can HMMs be extended to model joint change-points across multiple biological or time-series samples?
• RQ5 What role do model selection criteria like BIC and ICL play in determining the optimal number of change-points in HMM-based analysis?

Key findings

• The HMM framework enables efficient, linear-time computation of posterior state distributions using forward-backward recursions, making it scalable for long sequences.
• HMM-based inference supports uncertainty quantification through posterior marginal probabilities, which can be used to assess confidence in change-point locations.
• Extensions to joint segmentation across multiple samples allow for the identification of common change-points, improving detection power in comparative genomics.
• The integration of Bayesian networks and belief propagation enables exact inference and facilitates model extensions with complex dependencies.
• Model selection via BIC and ICL provides a principled approach to determining the number of segments and validating model fit.
• HMMs outperform non-parametric and binary segmentation methods in terms of accuracy and robustness when modeling heterogeneous data with known segment homogeneity.