[Paper Review] Gaussian Covariance faithful Markov Trees

Dhafer Malouche, Bala Rajaratnam|ArXiv.org|Dec 14, 2009

Bayesian Modeling and Causal Inference6 references3 citations

TL;DR

This paper proves that multivariate Gaussian distributions with tree-structured covariance graphs are necessarily faithful to their graphs, using a novel method that leverages path-based conditional independence analysis in subgraphs. The approach establishes faithfulness by showing non-zero partial correlation coefficients along critical paths, distinguishing it from existing methods used in concentration graph models.

ABSTRACT

A covariance graph is an undirected graph associated with a multivariate probability distribution of a given random vector where each vertex represents each of the different components of the random vector and where the absence of an edge between any pair of variables implies marginal independence between these two variables. Covariance graph models have recently received much attention in the literature and constitute a sub-family of graphical models. Though they are conceptually simple to understand, they are considerably more difficult to analyze. Under some suitable assumption on the probability distribution, covariance graph models can also be used to represent more complex conditional independence relationships between subsets of variables. When the covariance graph captures or reflects all the conditional independence statements present in the probability distribution the latter is said to be faithful to its covariance graph - though no such prior guarantee exists. Despite the increasingly widespread use of these two types of graphical models, to date no deep probabilistic analysis of this class of models, in terms of the faithfulness assumption, is available. Such an analysis is crucial in understanding the ability of the graph, a discrete object, to fully capture the salient features of the probability distribution it aims to describe. In this paper we demonstrate that multivariate Gaussian distributions that have trees as covariance graphs are necessarily faithful. The method of proof is original as it uses an entirely new approach and in the process yields a technique that is novel to the field of graphical models.

Motivation & Objective

• To establish faithfulness of multivariate Gaussian distributions to their covariance graphs, particularly when the graph is a tree.
• To address the lack of deep probabilistic analysis of faithfulness in covariance graph models despite their growing use.
• To develop a new, self-contained method for proving faithfulness that differs fundamentally from existing approaches used in concentration graph models.

Proposed method

• The proof uses a novel technique based on analyzing the inverse covariance matrix of subvectors induced by specific vertex sets in the tree.
• It identifies critical paths between sets A and B in the covariance graph G₀ that intersect the separating set S = V\(A∪B∪S), ensuring path connectivity.
• The method computes the partial correlation coefficient k_{u'v'|S} using a determinant-based formula involving submatrices of the covariance matrix.
• It shows that k_{u'v'|S} ≠ 0 by proving the expression in the formula is non-zero due to the structure of the path and the tree's acyclicity.
• The argument relies on the fact that the induced subgraph on {u',v'} ∪ S is itself a valid covariance graph for the corresponding subvector, enabling exact computation of conditional dependence.
• Contradiction is used: assuming conditional independence leads to k_{u'v'|S} = 0, which is shown to be impossible under the tree structure.

Experimental results

Research questions

• RQ1 Are multivariate Gaussian distributions with tree-structured covariance graphs always faithful to their graphs?
• RQ2 Can a new, self-contained method be developed to prove faithfulness in covariance graph models, distinct from methods used in concentration graph models?
• RQ3 Does the structure of a tree in a covariance graph guarantee that all conditional independence statements are encoded in the graph?
• RQ4 Can the faithfulness property be established without relying on the intersection or global Markov properties?
• RQ5 What is the role of path structure in determining non-zero partial correlations in Gaussian graphical models?

Key findings

• All multivariate Gaussian distributions whose covariance graphs are trees are faithful to their graphs, meaning the graph encodes all conditional independence relationships in the distribution.
• The faithfulness result is proven via a novel method that computes partial correlation coefficients using path-based determinant formulas in induced subgraphs.
• The method relies on the fact that the induced subgraph on any three vertices including a path is itself a valid covariance graph for the corresponding subvector.
• The proof shows that the partial correlation coefficient k_{u'v'|S} is non-zero when a path connects A and B through S, contradicting the assumption of conditional independence.
• This result contrasts with concentration graph models, where the same approach does not apply due to the non-preservation of subgraph structure in conditioning.
• The approach is fundamentally different from prior methods used in concentration graph models and opens new avenues for faithfulness analysis in other graphical model classes.