[Paper Review] A partial orthogonalization method for simulating covariance and concentration graph matrices

Irene Córdoba, Gherardo Varando+2 more|arXiv (Cornell University)|Jul 9, 2018

Bayesian Modeling and Causal Inference21 references4 citations

TL;DR

This paper proposes a partial orthogonalization method to generate symmetric positive definite (SPD) covariance and concentration matrices that preserve strong link strengths in graphical models, overcoming the issue of weak off-diagonal entries common in diagonally dominant matrices. By enforcing row-wise orthogonality for missing edges in the graph, the method ensures stable link strength across varying sparsity levels, significantly improving structure recovery in learning algorithms across synthetic and real-world validations.

ABSTRACT

Structure learning methods for covariance and concentration graphs are often validated on synthetic models, usually obtained by randomly generating: (i) an undirected graph, and (ii) a compatible symmetric positive definite (SPD) matrix. In order to ensure positive definiteness in (ii), a dominant diagonal is usually imposed. However, the link strengths in the resulting graphical model, determined by off-diagonal entries in the SPD matrix, are in many scenarios extremely weak. Recovering the structure of the undirected graph thus becomes a challenge, and algorithm validation is notably affected. In this paper, we propose an alternative method which overcomes such problem yet yielding a compatible SPD matrix. We generate a partially row-wise-orthogonal matrix factor, where pairwise orthogonal rows correspond to missing edges in the undirected graph. In numerical experiments ranging from moderately dense to sparse scenarios, we obtain that, as the dimension increases, the link strength we simulate is stable with respect to the structure sparsity. Importantly, we show in a real validation setting how structure recovery is greatly improved for all learning algorithms when using our proposed method, thereby producing a more realistic comparison framework.

Motivation & Objective

• To address the limitation of diagonally dominant matrices in structure learning algorithm validation, where off-diagonal entries become too weak as dimension increases.
• To develop a method that generates symmetric positive definite matrices compatible with an undirected graph structure while maintaining strong, stable link strengths.
• To improve the realism and reliability of synthetic benchmarks for evaluating covariance and concentration graph learning algorithms.
• To demonstrate that algorithm performance rankings and conclusions can be misleading when based on diagonally dominant matrices.
• To provide a computationally efficient, reproducible alternative to diagonal dominance for generating SPD matrices in graphical model validation.

Proposed method

• The method constructs a matrix factor where rows corresponding to non-adjacent nodes (missing edges) in the undirected graph are made pairwise orthogonal.
• It uses a modified Gram-Schmidt process to enforce partial orthogonality, preserving the desired graph structure.
• The resulting matrix factor is then transformed into a symmetric positive definite matrix via outer product, ensuring compatibility with the graph structure.
• The approach avoids diagonal dominance by design, thus preventing the suppression of off-diagonal entries.
• The method is implemented in R and released as the gmat package, supporting both partial orthogonalization and diagonal dominance for comparison.
• The implementation allows for efficient generation of matrices across varying sparsity levels and dimensions.

Experimental results

Research questions

• RQ1 Does the partial orthogonalization method produce covariance and concentration matrices with stronger and more stable link strengths compared to diagonally dominant matrices?
• RQ2 How does the performance of structure learning algorithms change when validated on matrices generated by the proposed method versus diagonal dominance?
• RQ3 Can the proposed method improve structure recovery in both moderately dense and sparse graphical models?
• RQ4 What impact does matrix generation method have on the true positive rate and true discovery rate of learning algorithms?
• RQ5 Does the proposed method yield more reliable and realistic performance comparisons in real-world validation settings?

Key findings

• When using diagonally dominant matrices, all structure learning algorithms achieved near-zero true positive rates for dense graphs (p=100, d=0.25), indicating poor recovery.
• With the proposed partial orthogonalization method, several algorithms achieved a true positive rate of approximately 0.45, demonstrating significant improvement in structure recovery.
• All learning algorithms showed higher true discovery rates when validated on matrices generated by the proposed method compared to diagonal dominance.
• Partial least squares regression, previously deemed ineffective in dense scenarios under diagonal dominance, showed reasonable performance when using the new method, challenging prior conclusions.
• The method maintains stable link strength with respect to graph sparsity, even as dimension increases, unlike diagonal dominance which suppresses off-diagonal entries.
• The real-world validation example demonstrates that algorithm rankings and performance assessments can be fundamentally altered depending on the matrix generation method used.