[Paper Review] Latent variable model selection for Gaussian conditional random fields

Benjamin Frot, Luke Jostins+1 more|arXiv (Cornell University)|Dec 20, 2015

Gene expression and cancer classification35 references3 citations

TL;DR

This paper proposes a low-rank plus sparse decomposition method for learning Gaussian conditional random fields in the presence of latent variables, using regularized maximum-likelihood estimation. The approach achieves sparsistent graph recovery and outperforms existing methods in simulations and real genetic data applications, showing improved biological relevance and replication across cohorts.

ABSTRACT

We consider the problem of learning a conditional Gaussian graphical model in the presence of latent variables. Building on recent advances in this field, we suggest a method that decomposes the parameters of a conditional Markov random field into the sum of a sparse and a low-rank matrix. We derive convergence bounds for this estimator and show that it is well-behaved in the high-dimensional regime as well as "sparsistent" (i.e. capable of recovering the graph structure). We then show how proximal gradient algorithms and semi-definite programming techniques can be employed to fit the model to thousands of variables. Through extensive simulations, we illustrate the conditions required for identifiability and show that there is a wide range of situations in which this model performs significantly better than its counterparts, for example, by accommodating more latent variables. Finally, the suggested method is applied to two datasets comprising individual level data on genetic variants and metabolites levels. We show our results replicate better than alternative approaches and show enriched biological signal.

Motivation & Objective

• To address the challenge of learning conditional Gaussian graphical models when latent variables confound observed dependencies.
• To develop a method that jointly models observed variables conditionally on measured covariates while accounting for unobserved latent factors.
• To ensure consistency and sparsistency in high-dimensional settings where p > n.
• To improve model identifiability and performance over existing methods in realistic genetic and omics data scenarios.

Proposed method

• Decomposes the inverse covariance matrix of the response variables into a sparse component (representing direct conditional dependencies) and a low-rank component (representing the effect of latent variables).
• Uses a regularized maximum-likelihood estimator with a penalty combining l1-norm (for sparsity) and nuclear norm (for low-rank structure).
• Employs proximal gradient algorithms and semidefinite programming to optimize the objective function efficiently, scaling to thousands of variables.
• Applies the alternating direction method of multipliers (ADMM) to solve the optimization problem in a distributed and scalable manner.
• Incorporates stability selection with complementary pairs to enhance robustness to tuning parameter choice and improve error control.
• Derives theoretical convergence bounds and establishes consistency and sparsistency under suitable identifiability conditions.

Experimental results

Research questions

• RQ1 Can a conditional Gaussian graphical model with latent variables be consistently estimated in high-dimensional settings where p > n?
• RQ2 Under what conditions is the low-rank plus sparse decomposition identifiable in the presence of latent confounders?
• RQ3 How does the proposed method compare to existing approaches in terms of graph structure recovery and biological relevance on real genetic data?
• RQ4 To what extent does the model improve replication of results across independent cohorts in genetic studies?
• RQ5 How robust is the method to tuning parameter selection, and can stability selection mitigate sensitivity?

Key findings

• The proposed estimator is consistent and sparsistent in the high-dimensional regime under appropriate identifiability conditions.
• The method significantly outperforms existing approaches in simulations, especially when multiple latent variables are present.
• In the ALSPAC genetic dataset, LSCGGM achieved the highest replication rates across mother and child cohorts, with higher values than competing methods in stable parameter regions.
• The model replicated biological signals more effectively than alternatives, as validated by independent sources and pathway enrichment analysis.
• The method showed reduced sensitivity to the tuning parameter γ compared to lasso-based estimators, enhancing practical usability.
• Stability selection with complementary pairs improved robustness and provided error control, supporting the generation of reliable causal hypotheses from high-dimensional data.