[Paper Review] A Probability Density Theory for Spin-Glass Systems

Gavin S. Hartnett, Masoud Mohseni|arXiv (Cornell University)|Jan 3, 2020

Theoretical and Computational Physics31 references4 citations

TL;DR

This paper introduces a continuous probability density theory for spin-glass systems by mapping discrete Ising spin models to a continuous Hamiltonian density over real variables, enabling application of deep generative models like normalizing flows. The key contribution is identifying a temperature-driven transition from convex to non-convex energy landscapes that precedes the spin-glass phase transition, without relying on quenched disorder averaging or mean-field approximations.

ABSTRACT

Spin-glass systems are universal models for representing many-body phenomena in statistical physics and computer science. High quality solutions of NP-hard combinatorial optimization problems can be encoded into low energy states of spin-glass systems. In general, evaluating the relevant physical and computational properties of such models is difficult due to critical slowing down near a phase transition. Ideally, one could use recent advances in deep learning for characterizing the low-energy properties of these complex systems. Unfortunately, many of the most promising machine learning approaches are only valid for distributions over continuous variables and thus cannot be directly applied to discrete spin-glass models. To this end, we develop a continuous probability density theory for spin-glass systems with arbitrary dimensions, interactions, and local fields. We show how our formulation geometrically encodes key physical and computational properties of the spin-glass in an instance-wise fashion without the need for quenched disorder averaging. We show that our approach is beyond the mean-field theory and identify a transition from a convex to non-convex energy landscape as the temperature is lowered past a critical temperature. We apply our formalism to a number of spin-glass models including the Sherrington-Kirkpatrick (SK) model, spins on random Erdős-Rényi graphs, and random restricted Boltzmann machines.

Motivation & Objective

• To develop a continuous probability density formulation for discrete spin-glass systems to enable application of deep generative models such as normalizing flows.
• To characterize low-energy and critical properties of spin-glass systems without relying on quenched disorder averaging or mean-field approximations.
• To identify geometric and topological features of the energy landscape that signal the onset of spin-glass behavior.
• To bridge the gap between discrete spin-glass models and continuous machine learning techniques, particularly in the context of NP-hard optimization problems.

Proposed method

• Mapping discrete Ising spin configurations to continuous variables via a continuous Hamiltonian density $ \mathcal{H}_{\beta}(x) $, transforming the discrete partition function into a continuous integral.
• Defining a Boltzmann distribution over continuous variables: $ p(x) = e^{-\beta \mathcal{H}_{\beta}(x)} / Z_x $, with $ Z_x = \int d^N x \, e^{-\beta \mathcal{H}_{\beta}(x)} $.
• Using diagrammatic perturbation theory to compute the disorder-averaged logarithm of the partition function, identifying contributions from cyclic diagrams (e.g., polygons) that signal non-analytic behavior.
• Deriving a critical temperature $ T_{\text{convex}} $ at which the energy landscape transitions from convex to non-convex, based on the Hessian of the continuous Hamiltonian.
• Applying the formalism to the Sherrington-Kirkpatrick model, Erdős-Rényi random graphs, and restricted Boltzmann machines to validate generality.
• Analyzing the 2D Ising model to compare $ T_{\text{convex}} $, $ T_{\text{mean-field}} $, and the true critical temperature $ T_{\text{crit}} $, showing $ T_{\text{convex}} $ occurs before the phase transition.

Experimental results

Research questions

• RQ1 Can a continuous probability density formulation be constructed for discrete spin-glass systems to enable deep generative modeling?
• RQ2 Does the convexity of the energy landscape in the continuous formulation signal a physical phase transition or a precursor to spin-glass ordering?
• RQ3 How does the critical temperature $ T_{\text{convex}} $ derived from the continuous Hamiltonian compare to the true spin-glass phase transition temperature?
• RQ4 Can this formalism capture non-mean-field behavior and non-perturbative features of spin-glass systems without quenched averaging?
• RQ5 What is the role of cyclic diagrams (e.g., polygons) in the high-temperature expansion of the partition function, and do they signal the breakdown of perturbation theory?

Key findings

• The continuous Hamiltonian density formulation successfully maps discrete spin-glass models to continuous probability distributions, enabling application of continuous generative models like normalizing flows.
• A transition from convex to non-convex energy landscape occurs at $ T_{\text{convex}} $, which is found to precede the true spin-glass phase transition in the 2D Ising model.
• In the 2D Ising model, $ T_{\text{convex}} = 2 \times T_{\text{mean-field}} \approx 8\mathcal{J} $, while the true critical temperature is $ T_{\text{crit}} \approx 2.2692\mathcal{J} $, confirming $ T_{\text{convex}} $ occurs well before the phase transition.
• The high-temperature expansion of the partition function reveals contributions from cyclic diagrams (e.g., polygons), which can be resummed into a logarithmic singularity at $ \beta^2 \mathcal{J}^2 = 1 $, indicating breakdown of perturbation theory.
• The disorder-averaged partition function derived via the continuous formalism reproduces the known TAP result, validating the approach.
• The transition at $ T_{\text{convex}} $ is not a physical phase transition, as both $ Z_s $ and $ Z_x $ remain analytic and well-behaved at this temperature, indicating it is a geometric feature of the continuous formulation.