[Paper Review] Efficient Neighborhood Selection for Gaussian Graphical Models

Yingxiang Yang, Jalal Etesami+1 more|arXiv (Cornell University)|Sep 22, 2015

Bayesian Modeling and Causal Inference9 references3 citations

TL;DR

This paper proposes two efficient, structure-consistent algorithms for neighborhood selection in Gaussian graphical models: a forward-backward greedy algorithm based on conditional mutual information and a threshold-based algorithm for walk-summable models. The key contribution is a theoretical lower bound on the conditional covariance between a node and its undiscovered neighbors, enabling provably consistent and efficient neighborhood recovery without tuning parameters in the threshold-based method.

ABSTRACT

This paper addresses the problem of neighborhood selection for Gaussian graphical models. We present two heuristic algorithms: a forward-backward greedy algorithm for general Gaussian graphical models based on mutual information test, and a threshold-based algorithm for walk summable Gaussian graphical models. Both algorithms are shown to be structurally consistent, and efficient. Numerical results show that both algorithms work very well.

Motivation & Objective

• To develop efficient and structurally consistent algorithms for neighborhood selection in Gaussian graphical models.
• To establish a theoretical lower bound on the conditional covariance between a node and its undiscovered neighbors in walk-summable models.
• To eliminate reliance on tuning parameters in neighborhood selection by leveraging the conditional covariance lower bound.
• To improve computational efficiency and convergence speed compared to existing greedy methods like forward-backward selection.
• To demonstrate structural consistency and sample efficiency through numerical evaluation on synthetic graphs.

Proposed method

• Proposes a forward-backward greedy algorithm that iteratively selects neighbors based on conditional mutual information and prunes low-likelihood neighbors using a threshold.
• Introduces a threshold-based algorithm for walk-summable Gaussian graphical models that leverages a lower bound on the absolute value of conditional covariance between nodes and their undiscovered neighbors.
• Uses a pruning threshold of $10^{-3}$ in Algorithm 3 to control pseudo-neighborhood size, especially when sample size is small.
• Employs a stopping criterion for forward selection based on a threshold $\epsilon_s = 8c\rho\eta d\log(m)/(kC_{\text{min}})$, where $c$ is a tuning parameter.
• Applies symmetry-based pruning in the forward phase to reduce computational overhead.
• Derives upper bounds on the ratio between the selected pseudo-neighborhood size and the true degree, and upper bounds the number of algorithm iterations.

Experimental results

Research questions

• RQ1 Can neighborhood selection in Gaussian graphical models be achieved efficiently using an information-theoretic criterion like conditional mutual information?
• RQ2 Is there a natural influence measure—such as conditional covariance—that allows for a lower bound on the influence from undiscovered neighbors?
• RQ3 Can the maximal conditional covariance between a node and its undiscovered neighbors be bounded away from zero in walk-summable models?
• RQ4 Does a threshold-based algorithm exist that ensures structural consistency without requiring tuning of parameters?
• RQ5 How does the performance of the proposed threshold-based algorithm compare to tuned greedy methods in terms of convergence and pseudo-neighborhood size?

Key findings

• The threshold-based algorithm achieves performance comparable to the best-tuned forward-backward greedy algorithm across all tested values of the tuning parameter $c$.
• The performance of the forward-backward greedy algorithm is highly sensitive to the choice of the tuning parameter $c$, while the proposed threshold-based algorithm requires no such tuning.
• When the number of samples is large, both algorithms efficiently select the pseudo-neighborhood, with the threshold-based method matching the best performance of the tuned greedy algorithm.
• The threshold-based algorithm selects a larger pseudo-neighborhood when the number of samples is small, due to the fixed pruning threshold of $10^{-3}$.
• The theoretical lower bound on conditional covariance ensures that the algorithm can reliably distinguish true neighbors from non-neighbors, enabling structural consistency.
• The number of iterations and the ratio of pseudo-neighborhood size to true degree are both upper bounded, confirming computational efficiency.