[Paper Review] Towards meaningful physics from generative models

M. Cristoforetti, Giuseppe Jurman+2 more|arXiv (Cornell University)|May 26, 2017

Computational Physics and Python Applications19 references16 citations

TL;DR

This paper proposes a physics-informed variational autoencoder (H/G-VAE) that enhances generative models for Monte Carlo-simulated physical systems, enabling accurate detection of the Kosterlitz-Thouless (KT) phase transition in the 2D XY model and improving reconstruction of topological defects like vortices. By incorporating energy and Gaussian fluctuation constraints, the model achieves meaningful physical content in generated configurations while preserving phase transition identification capabilities.

ABSTRACT

In several physical systems, important properties characterizing the system itself are theoretically related with specific degrees of freedom. Although standard Monte Carlo simulations provide an effective tool to accurately reconstruct the physical configurations of the system, they are unable to isolate the different contributions corresponding to different degrees of freedom. Here we show that unsupervised deep learning can become a valid support to MC simulation, coupling useful insights in the phases detection task with good reconstruction performance. As a testbed we consider the 2D XY model, showing that a deep neural network based on variational autoencoders can detect the continuous Kosterlitz-Thouless (KT) transitions, and that, if endowed with the appropriate constrains, they generate configurations with meaningful physical content.

Motivation & Objective

• To address the challenge of isolating contributions from specific physical degrees of freedom—particularly topological excitations—within Monte Carlo-generated configurations.
• To improve generative modeling of statistical physics systems by embedding physical constraints into deep learning architectures.
• To enable unsupervised deep learning to detect continuous phase transitions, such as the Kosterlitz-Thouless transition, without relying on predefined order parameters.
• To validate that physics-informed modifications to variational autoencoders enhance reconstruction fidelity and physical relevance of generated configurations.

Proposed method

• The study employs a standard Variational Autoencoder (VAE) as a baseline for learning latent representations of 2D XY model configurations from Monte Carlo simulations.
• A physics-informed variant, H/G-VAE, is introduced by adding two constraints: an energy-based loss term (H-VAE) and a Gaussian fluctuation term (G-VAE) to regularize the latent space.
• The energy loss term enforces consistency between generated configurations and the system’s Hamiltonian, improving physical plausibility.
• The Gaussian fluctuation term enhances the model’s ability to reproduce topological defects such as vortices, which are critical to the KT transition.
• Latent space analysis is used to detect phase transitions by observing changes in the distribution of latent variables across different inverse temperatures (β).
• The model’s performance is evaluated using physical observables: energy, magnetization, and vortex density, compared against true Monte Carlo values.

Experimental results

Research questions

• RQ1 Can a deep generative model detect the continuous Kosterlitz-Thouless phase transition in the 2D XY model without relying on a predefined order parameter?
• RQ2 How do physics-informed constraints in the VAE loss function affect the reconstruction quality of Monte Carlo configurations?
• RQ3 To what extent can a VAE model reproduce topological defects such as vortices, which are central to the KT transition?
• RQ4 Does the inclusion of physical constraints in the VAE architecture improve the identification of critical temperatures compared to a standard VAE?
• RQ5 Can the latent space of a VAE reveal distinct phases of the system, even when the model is not explicitly trained to classify them?

Key findings

• The H/G-VAE model successfully detects the Kosterlitz-Thouless transition by showing a clear separation in the latent space distribution at β ≈ 0.1, which corresponds to the true critical temperature.
• The inclusion of the energy loss term (H-VAE) improves the model’s ability to identify phase transitions, aligning the critical point with the one derived from magnetic susceptibility.
• The Gaussian fluctuation term (G-VAE) significantly enhances the reconstruction of vortex density, bringing it closer to the true Monte Carlo values, especially near the critical region.
• Reconstruction results show that the G-VAE produces configurations with more realistic disorder and vortex structures, while the standard VAE overestimates order.
• The H/G-VAE maintains high performance in reconstructing energy and magnetization, though discrepancies remain, suggesting room for further model specialization.
• Visual comparisons confirm that the G-VAE produces qualitatively more realistic configurations than the standard VAE, particularly in capturing the disordered phase at low β.