[Paper Review] Learning Clique Forests

Guido Previde Massara, Tomaso Aste|arXiv (Cornell University)|May 6, 2019

Statistical Methods and Inference71 references4 citations

TL;DR

This paper introduces the Maximally Filtered Clique Forest (MFCF), a novel algorithm for learning sparse, chordal graphical models from data by recursively expanding a forest of cliques using a topologically invariant clique expansion operator. The method produces decomposable Markov random fields that enable efficient exact inference and outperforms Graphical Lasso and shrinkage methods in covariance selection tasks, particularly with small samples.

ABSTRACT

We propose a topological learning algorithm for the estimation of the conditional dependency structure of large sets of random variables from sparse and noisy data. The algorithm, named Maximally Filtered Clique Forest (MFCF), produces a clique forest and an associated Markov Random Field (MRF) by generalising Prim's minimum spanning tree algorithm. To the best of our knowledge, the MFCF presents three elements of novelty with respect to existing structure learning approaches. The first is the repeated application of a local topological move, the clique expansion, that preserves the decomposability of the underlying graph. Through this move the decomposability and calculation of scores is performed incrementally at the variable (rather than edge) level, and this provides better computational performance and an intuitive application of multivariate statistical tests. The second is the capability to accommodate a variety of score functions and, while this paper is focused on multivariate normal distributions, it can be directly generalised to different types of statistics. Finally, the third is the variable range of allowed clique sizes which is an adjustable topological constraint that acts as a topological penalizer providing a way to tackle sparsity at $l_0$ semi-norm level; this allows a clean decoupling of structure learning and parameter estimation. The MFCF produces a representation of the clique forest, together with a perfect ordering of the cliques and a perfect elimination ordering for the vertices. As an example we propose an application to covariance selection models and we show that the MCFC outperforms the Graphical Lasso for a number of classes of matrices.

Motivation & Objective

• To develop a general, scalable method for learning complex network structures from data that ensures computational tractability through chordal graph formation.
• To generalize the TMFG algorithm by allowing arbitrary clique sizes and multi-clique separators, overcoming structural rigidity in prior methods.
• To enable efficient exact inference in graphical models by constructing decomposable models via clique forests with bounded treewidth.
• To provide a flexible framework where gain functions can be customized for diverse applications, including covariance selection and topological data analysis.
• To explore the potential of clique forests as a foundation for persistent homology and dynamic network updates in evolving data.

Proposed method

• The MFCF algorithm uses a clique expansion operator that preserves the clique forest structure and ensures topological invariance during recursive node addition.
• New nodes are attached by maximizing a gain function that quantifies structural improvement, such as conditional independence or likelihood.
• The resulting clique forests are chordal graphs, guaranteeing decomposability and enabling efficient inference via junction tree algorithms.
• The method supports customizable clique sizes and separator multiplicities, allowing control over model complexity and treewidth.
• The algorithm generalizes Prim’s MST algorithm and can be adapted to Kruskal-like variants, though initial results suggest suboptimal performance due to early clustering.
• A novel 'clique tree target' shrinkage estimator is proposed, improving performance on small-sample covariance estimation.

Experimental results

Research questions

• RQ1 Can a general-purpose network learning algorithm be designed that ensures chordality and decomposability while remaining flexible to diverse data types and gain functions?
• RQ2 How does the MFCF method compare to established methods like Graphical Lasso in covariance selection tasks, especially under small sample conditions?
• RQ3 To what extent can the clique forest structure support topological data analysis, including persistent homology and hierarchical network tracking?
• RQ4 What are the implications of using a clique expansion operator for maintaining model decomposability and enabling exact inference?
• RQ5 Can the MFCF framework be extended to incorporate expert knowledge or dynamic data updates through pruning or reconfiguration?

Key findings

• The MFCF algorithm consistently outperforms Graphical Lasso and standard shrinkage estimators in covariance selection tasks, particularly with small sample sizes.
• The resulting clique forests are chordal, ensuring that the associated graphical models are decomposable and support exact inference via junction tree algorithms.
• The clique expansion operator maintains the clique forest structure and enables recursive, topologically invariant growth of the network.
• The method generalizes TMFG by allowing arbitrary clique sizes and multi-clique separators, significantly increasing structural flexibility.
• The proposed 'clique tree target' shrinkage estimator yields encouraging results in small-sample settings, suggesting improved estimation accuracy.
• The MFCF framework supports dynamic updates and pruning, as the clique forest structure allows identification of removable edges without disrupting geometry.