[Paper Review] Learning phase transitions: comparing PCA and SVM

R. M. Woloshyn|arXiv (Cornell University)|May 20, 2019

Neural Networks and Applications3 references4 citations

TL;DR

This paper compares principal component analysis (PCA) and support vector machines (SVM) for detecting and characterizing phase transitions in two-dimensional classical spin models, including the Blume-Capel and Biquadratic Spin-1 Ising models. Using raw Monte Carlo spin configurations as input, PCA identifies dominant features like magnetization without prior knowledge, while SVM trained on ordered and disordered phases provides a decision function whose standard deviation acts as a proxy for susceptibility, enabling quantitative critical temperature estimation at T_c = 0.1655(5) for the BSI model.

ABSTRACT

A comparison of results from principal component analysis and support vector machine calculations is made for a variety of phase transitions in two-dimensional classical spin models.

Motivation & Objective

• To extend SVM analysis beyond the Ising model to other classical spin systems like the Blume-Capel and Biquadratic Spin-1 Ising models.
• To compare the performance and insights from unsupervised PCA and supervised SVM in detecting phase transitions without prior knowledge of order parameters.
• To evaluate whether the standard deviation of the SVM decision function can serve as a reliable proxy for susceptibility in identifying critical points.
• To estimate the critical temperature of the Biquadratic Spin-1 Ising model using SVM-based scaling analysis.

Proposed method

• PCA is applied to raw spin configurations from Monte Carlo simulations, decomposing data into principal components via singular value decomposition of centered data matrices.
• The leading principal components are quantified using the average absolute value over configurations at fixed parameters to identify dominant order parameters like magnetization.
• SVM is trained as a binary classifier on spin configurations from two distinct simulation regimes (e.g., low and high temperature) to distinguish ordered and disordered phases.
• A homogeneous quadratic kernel is used to compute the decision function d(x), which is interpolated across the parameter space to probe intermediate states.
• The standard deviation of the decision function σ_d is scaled by L² to approximate the susceptibility and locate phase transition peaks.
• Critical temperature T_c is estimated by extrapolating peak positions of L²σ_d across lattice sizes L using a T_c(L) = T_c + c/L fit.

Experimental results

Research questions

• RQ1 Can PCA reliably identify the dominant order parameter (e.g., magnetization) in phase transitions without prior domain knowledge?
• RQ2 Does the SVM decision function's standard deviation serve as a valid proxy for susceptibility in detecting phase transitions?
• RQ3 Can the SVM-based method quantitatively estimate the critical temperature T_c for models beyond the Ising model?
• RQ4 How do the PCA and SVM results compare in distinguishing between genuine phase transitions and crossovers?

Key findings

• PCA successfully identifies magnetization as the leading principal component in the Ising and Blume-Capel models without prior input, confirming its ability to extract physical order parameters from raw data.
• For the Biquadratic Spin-1 Ising model, the second quantified principal component from PCA reproduces ⟨s²⟩, indicating its sensitivity to spin-squared correlations.
• The SVM decision function's standard deviation, when scaled by L², peaks at T ≈ 1.7 for the BSI model, mirroring the behavior of the true susceptibility.
• The critical temperature for the BSI model at K=1, J=0.1 is estimated as T_c = 0.1655(5), consistent with prior results and obtained via finite-size scaling of L²σ_d.
• The SVM-based method enables quantitative analysis of phase transitions without assuming an order parameter, demonstrating its utility in complex spin systems.
• Both PCA and SVM reveal distinct transition behaviors in the Blume-Capel and BSI models, with PCA offering qualitative insight and SVM enabling quantitative critical point estimation.