[Paper Review] The density of states from first principles

Roberto Pellegrini, Kurt Langfeld+2 more|arXiv (Cornell University)|Nov 3, 2014

Quantum Chromodynamics and Particle Interactions3 references3 citations

TL;DR

This paper presents a novel, mathematically rigorous algorithm to compute the density of states in continuous systems using a stochastic root-finding approach based on the Robbins-Monro method. It enables efficient computation of partition functions and observables in statistical and quantum field theories, validated by accurate results in 4d U(1) lattice gauge theory, including precise determination of the pseudo-critical coupling and specific heat peak.

ABSTRACT

We present a novel algorithm to compute the density of states, which is proven to converge to the correct result. The algorithm is very general and can be applied to a wide range of models, in the frameworks of Statistical Mechanics and Lattice Gauge Theory. All the thermal or quantum expectation values can then be obtained by a simple integration of the density of states. As an application, a numerical study of 4d U(1) compact lattice gauge theory is presented.

Motivation & Objective

• To overcome the limitations of standard Monte Carlo methods in non-probabilistic path integrals, such as at finite density or in theories like QCD.
• To develop a general-purpose algorithm for computing the density of states in continuous systems, applicable to both statistical mechanics and lattice gauge theories.
• To enable the computation of thermal and quantum expectation values via simple integration over the density of states.
• To provide a mathematically proven, convergent method that avoids the sign problem and is suitable for oscillatory or non-probabilistic path integrals.

Proposed method

• The method uses a piecewise linear approximation of log ρ(E) around energy E₀, parameterized by coefficient a(E₀) = d/dE log ρ(E) |_{E=E₀}.
• It formulates the problem as a root-finding task: find a* such that the truncated energy deviation ΔE(a*) = 0, which corresponds to a flat distribution over the energy interval.
• The Robbins-Monro stochastic algorithm is employed to iteratively update aₙ using ΔE(aₙ) from Monte Carlo sampling with a weight proportional to exp(−aE) within a narrow energy window.
• The step size is set as cₙ = 3/( (n+1)Δ² ) to minimize asymptotic variance, ensuring almost sure convergence to the true a*.
• Ergodicity is improved by using overlapping energy intervals and implementing a parallel tempering-like swap mechanism between simulations in overlapping regions.
• The method computes ρ(E) by integrating over energy intervals, allowing reconstruction of the partition function Z(β) = ∫ dE ρ(E) e^{-βE} and observables via ⟨O⟩ = Z⁻¹ ∫ dE O(E) ρ(E) e^{-βE}.

Experimental results

Research questions

• RQ1 Can a robust, convergent algorithm be developed for computing the density of states in continuous systems, particularly in models where standard Monte Carlo fails?
• RQ2 How can the density of states be computed efficiently in theories with non-probabilistic path integrals, such as finite-density QCD?
• RQ3 What is the optimal stochastic root-finding strategy for estimating the derivative of log ρ(E) in a numerically stable and convergent manner?
• RQ4 How can ergodicity be maintained when sampling in narrow energy windows in rugged energy landscapes?
• RQ5 To what extent can this method reproduce known critical properties in lattice gauge theories, such as the pseudo-critical coupling and specific heat peak?

Key findings

• The algorithm successfully computes the density of states in 4d U(1) compact lattice gauge theory on a 12⁴ lattice, with plaquette values matching those from heat-bath and multi-canonical simulations.
• The pseudo-critical coupling β_c was determined as 10111331(21), in excellent agreement with the established value from Borgs-Kotecký finite-size scaling.
• The specific heat peak was located for lattice sizes from 8⁴ to 20⁴, with results consistent with prior work, including a peak value of 1.011006(2) at 20⁴.
• The method achieved convergence with a 512-interval energy grid over [0.59, 0.69], with errors estimated via bootstrap analysis over 20 independent simulations.
• The longest simulation on a 20⁴ lattice required 512×144 hours of CPU time, demonstrating feasibility for large-scale studies.
• The algorithm correctly captured the double-peak structure in the action probability density P(E)₆ at β near β_c, confirming a weak first-order phase transition.