[Paper Review] Spectral Learning of Large Structured HMMs for Comparative Epigenomics

Chicheng Zhang, Jimin Song+2 more|arXiv (Cornell University)|Jun 4, 2015

Genomics and Chromatin Dynamics17 references3 citations

TL;DR

This paper proposes Spectral-Tree, a novel spectral learning algorithm for large structured Hidden Markov Models (HMMs) with a tree-structured dependency among hidden states across multiple human cell types. By exploiting the tree structure through root-to-leaf path decomposition, Skeletensor construction, and Product Projections, the method achieves polynomial-time complexity in the number of cell types, enabling efficient and accurate learning of chromatin state segmentation from multi-cell-type epigenomic data, with demonstrated improvements over EM and individual HMM learning.

ABSTRACT

We develop a latent variable model and an efficient spectral algorithm motivated by the recent emergence of very large data sets of chromatin marks from multiple human cell types. A natural model for chromatin data in one cell type is a Hidden Markov Model (HMM); we model the relationship between multiple cell types by connecting their hidden states by a fixed tree of known structure. The main challenge with learning parameters of such models is that iterative methods such as EM are very slow, while naive spectral methods result in time and space complexity exponential in the number of cell types. We exploit properties of the tree structure of the hidden states to provide spectral algorithms that are more computationally efficient for current biological datasets. We provide sample complexity bounds for our algorithm and evaluate it experimentally on biological data from nine human cell types. Finally, we show that beyond our specific model, some of our algorithmic ideas can be applied to other graphical models.

Motivation & Objective

• To address the computational infeasibility of EM-based learning for large, structured HMMs modeling multiple human cell types.
• To overcome the exponential time and space complexity of naive spectral methods when applied to HMMs with tree-structured hidden states.
• To develop a computationally efficient, scalable spectral algorithm that leverages biological knowledge of cell lineage trees for comparative epigenomics.
• To provide theoretical sample complexity bounds and empirical validation on real biological data from nine human cell types.
• To generalize the method to other graphical models through reusable algorithmic components like Product Projections.

Proposed method

• The model is a tree-structured HMM where each cell type's hidden state depends on its parent in a known phylogenetic tree, with independent observations per node.
• The algorithm processes each root-to-leaf path separately, reducing the problem to tensor decomposition on low-depth paths.
• Skeletensor construction is introduced to avoid full tensor computation by symmetrizing tensors using rank-revealing matrices, reducing dimensionality to that of a single node.
• Product Projections compress the observation tensor by exploiting conditional independence, avoiding full tensor construction and reducing complexity to the number of hidden states.
• The method uses tensor decomposition and matrix inversion techniques to estimate transition and emission matrices with bounded error under mild rank conditions.
• Theoretical guarantees are derived using matrix perturbation theory, including Weyl’s and Wedin’s theorems, to bound estimation error.

Experimental results

Research questions

• RQ1 Can spectral learning be made scalable for large, structured HMMs with tree-organized hidden states across multiple biological samples?
• RQ2 Does exploiting the tree structure of cell lineage relationships improve the accuracy and efficiency of chromatin state segmentation compared to independent HMMs or EM?
• RQ3 Can tensor decomposition methods be adapted to avoid exponential complexity in the number of cell types in multi-view graphical models?
• RQ4 What is the sample complexity of the proposed spectral algorithm under realistic biological data assumptions?
• RQ5 Can the core algorithmic innovations, such as Product Projections, be generalized to other latent variable models beyond HMMs?

Key findings

• The Spectral-Tree algorithm achieves polynomial-time complexity in the number of cell types, avoiding the exponential blowup of naive spectral methods.
• On nine human cell types from ENCODE, the method outperforms variational EM and individual HMM learning in segmentation accuracy and parameter estimation.
• The algorithm provides theoretical sample complexity bounds, showing convergence to true parameters with high probability under mild rank conditions.
• Skeletensor construction reduces the effective tensor dimension to that of a single node, enabling efficient computation without full tensor storage.
• Product Projections allow compressed tensor estimation with complexity proportional to the number of hidden states, not observations, significantly improving scalability.
• Empirical results show improved estimation of transition and emission probabilities, with error bounds on the order of ε under appropriate sample size conditions.