[Paper Review] Learning Planar Ising Models

Jason K. Johnson, Praneeth Netrapalli+1 more|arXiv (Cornell University)|Nov 15, 2010

Bayesian Methods and Mixture Models24 references3 citations

TL;DR

This paper proposes a greedy algorithm to learn the best planar Ising model approximation for binary random variables using known pairwise correlations. By leveraging tractable inference in planar Ising models and efficient planarity testing, the method constructs a maximal planar graph that optimally fits the observed correlations, achieving exact recovery in simulations and modeling U.S. Senate voting patterns with high accuracy.

ABSTRACT

Inference and learning of graphical models are both well-studied problems in statistics and machine learning that have found many applications in science and engineering. However, exact inference is intractable in general graphical models, which suggests the problem of seeking the best approximation to a collection of random variables within some tractable family of graphical models. In this paper, we focus our attention on the class of planar Ising models, for which inference is tractable using techniques of statistical physics [Kac and Ward; Kasteleyn]. Based on these techniques and recent methods for planarity testing and planar embedding [Chrobak and Payne], we propose a simple greedy algorithm for learning the best planar Ising model to approximate an arbitrary collection of binary random variables (possibly from sample data). Given the set of all pairwise correlations among variables, we select a planar graph and optimal planar Ising model defined on this graph to best approximate that set of correlations. We demonstrate our method in some simulations and for the application of modeling senate voting records.

Motivation & Objective

• To develop a tractable method for learning graphical models when exact inference is infeasible in general models.
• To approximate a set of binary random variables with a planar Ising model that matches their pairwise correlations.
• To enable efficient learning of graphical structure and parameters using known techniques for planar graphs and Ising model inference.
• To apply the method to real-world data, such as U.S. Senate voting records, to uncover latent interaction structures.
• To extend the framework to non-zero-field outer-planar models for broader applicability.

Proposed method

• Use a greedy algorithm to iteratively add edges to a graph, selecting those that most improve the likelihood of matching observed pairwise correlations.
• Apply planarity testing and embedding algorithms (e.g., Chrobak and Payne) to ensure the resulting graph remains planar at each step.
• Leverage tractable Ising model inference via matrix determinant computation (Kac-Ward, Kasteleyn) to compute partition functions and likelihoods efficiently.
• Formulate the learning problem as maximizing the log-likelihood of the observed pairwise moments under the planar Ising model.
• For non-zero mean models, use an auxiliary node trick to map outer-planar models to equivalent zero-field planar models for tractable inference.
• Use Gibbs sampling to generate empirical data from known models for simulation validation.

Experimental results

Research questions

• RQ1 Can a greedy algorithm efficiently learn a planar Ising model that best approximates a given set of pairwise correlations among binary random variables?
• RQ2 How well does the method recover the true graphical structure when only pairwise moments are available?
• RQ3 Can the method be extended to model non-zero mean Ising models on outer-planar graphs with tractable inference?
• RQ4 How accurate is the learned model in capturing real-world dependencies, such as those in U.S. Senate voting patterns?
• RQ5 What is the impact of sample size on the accuracy of graphical model recovery?

Key findings

• With 10^5 samples, the algorithm achieved perfect recovery of the underlying planar graphical model in simulations.
• At 10^4 samples, the algorithm recovered the true model exactly for both zero-field planar and non-zero-field outer-planar Ising models.
• For the U.S. Senate voting data, the learned planar graph correctly captured party affiliations and caucusing patterns, such as Sanders and Lieberman connecting only to Democrats.
• The number of erroneous edges in the recovered model decreased significantly with increasing sample size, from 0.18 at 10^3 samples to zero at 10^4 samples.
• The method successfully identified McConnell as well-connected to Republicans and Reid as less central, consistent with known political dynamics.
• The algorithm's performance is robust and scalable, with exact recovery observed under sufficient sampling.