[Paper Review] Learning non-Gaussian graphical models via Hessian scores and triangular transport

Ricardo Baptista, Youssef Marzouk+2 more|arXiv (Cornell University)|Jan 8, 2021

Bayesian Modeling and Causal Inference64 references1 citations

TL;DR

This paper proposes a novel algorithm, sing, for learning the Markov structure of continuous, non-Gaussian graphical models by leveraging Hessian-based scores from joint log-density and triangular transport maps. It uses a deterministic coupling via transport maps to estimate density and exploit sparsity in the map to recover the true graph structure, even with biased density approximations, demonstrating consistent structure recovery on non-Gaussian and chaotic dynamical systems data.

ABSTRACT

Undirected probabilistic graphical models represent the conditional dependencies, or Markov properties, of a collection of random variables. Knowing the sparsity of such a graphical model is valuable for modeling multivariate distributions and for efficiently performing inference. While the problem of learning graph structure from data has been studied extensively for certain parametric families of distributions, most existing methods fail to consistently recover the graph structure for non-Gaussian data. Here we propose an algorithm for learning the Markov structure of continuous and non-Gaussian distributions. To characterize conditional independence, we introduce a score based on integrated Hessian information from the joint log-density, and we prove that this score upper bounds the conditional mutual information for a general class of distributions. To compute the score, our algorithm SING estimates the density using a deterministic coupling, induced by a triangular transport map, and iteratively exploits sparse structure in the map to reveal sparsity in the graph. For certain non-Gaussian datasets, we show that our algorithm recovers the graph structure even with a biased approximation to the density. Among other examples, we apply SING to learn the dependencies between the states of a chaotic dynamical system with local interactions.

Motivation & Objective

• To address the lack of consistent structure learning methods for continuous, non-Gaussian graphical models.
• To develop a score based on integrated Hessian information that upper bounds conditional mutual information for general non-Gaussian distributions.
• To enable sparse graph recovery by exploiting sparsity in a transport map that deterministically couples the data to a standard Gaussian.
• To demonstrate robustness to biased density approximations in non-Gaussian settings.
• To apply the method to real-world problems such as chaotic dynamical systems with local interactions.

Proposed method

• Proposes a Hessian-based score derived from the integrated squared Hessian of the joint log-density, which upper bounds conditional mutual information.
• Uses a triangular transport map to deterministically couple the data distribution to a standard Gaussian, enabling density estimation without Monte Carlo sampling.
• Estimates the transport map using polynomial chaos expansions with sparsity-promoting regularization to reveal structural sparsity.
• Applies a thresholding procedure on the Hessian score estimates, using asymptotic standard errors for statistical inference.
• Employs a coordinate-descent-like algorithm to iteratively refine the transport map and graph structure, leveraging sparsity in the map's Hessian.
• Uses a union bound over variable pairs to ensure asymptotic consistency in edge set recovery.

Experimental results

Research questions

• RQ1 Can a Hessian-based score consistently estimate conditional independence in non-Gaussian distributions?
• RQ2 Can a triangular transport map with sparse structure reveal the underlying graph sparsity in non-Gaussian data?
• RQ3 Does the method remain consistent even when the density estimate is biased?
• RQ4 Can the algorithm recover the true graph structure in chaotic dynamical systems with local interactions?
• RQ5 How does the method compare to existing approaches in terms of robustness and accuracy on non-Gaussian data?

Key findings

• The proposed Hessian score upper bounds conditional mutual information for a general class of non-Gaussian distributions, providing a valid score for structure learning.
• The algorithm achieves asymptotic consistency in edge set recovery: the probability of failing to recover the true graph tends to zero as sample size increases.
• The method successfully recovers the graph structure in non-Gaussian datasets even with biased density approximations, demonstrating robustness.
• On a chaotic dynamical system with local interactions, sing accurately recovers the true dependency structure from finite samples.
• Memory usage scales with dimension and polynomial degree, but remains manageable, with first-iteration memory use under 100 MB for d=12 and β=2.
• Theoretical analysis confirms that false positive and false negative rates converge to zero under regularity conditions, ensuring consistent structure learning.