[Paper Review] Dynamical Phase Transition in a Neural Network Model with Noise: an Exact Solution

Cristián Huepe, Maximino Aldana|arXiv (Cornell University)|Feb 22, 2002

Neural Networks and Applications11 references4 citations

TL;DR

This paper presents an exact analytical solution for a noise-driven dynamical phase transition in a Boolean neural network with random, quenched connections. By modeling the network's order parameter dynamics using a stochastic differential equation, the authors derive a second-order phase transition at a critical noise level ηc, with a critical exponent of 1/2, showing that organization persists below ηc and collapses into randomness above it, regardless of the specific weight distribution as long as it is non-symmetric.

ABSTRACT

The dynamical organization in the presence of noise of a Boolean neural network with random connections is analyzed. For low levels of noise, the system reaches a stationary state in which the majority of its elements acquire the same value. It is shown that, under very general conditions, there exists a critical value of the noise, below which the network remains organized and above which it behaves randomly. The existence and nature of the phase transition are computed analytically, showing that the critical exponent is 1/2. The dependence of the critical noise on the parameters of the network is obtained. These results are then compared with two numerical realizations of the network.

Motivation & Objective

• To investigate how noise affects the dynamical organization in a neural network with random, quenched connections.
• To determine whether a critical noise threshold ηc exists beyond which the system transitions from ordered to disordered behavior.
• To analytically compute the critical exponent and the dependence of ηc on network parameters such as connectivity K and weight distribution.
• To validate the analytical results against numerical simulations of two distinct network realizations.
• To establish the robustness of the phase transition under general conditions on the weight distribution.

Proposed method

• The network consists of N binary elements (σi = ±1) with K random, quenched connections per node, each weighted by independent random variables with a given probability density function P_c(x).
• A stochastic update rule is introduced where each node updates with probability 1−η according to the sign of the weighted sum of its inputs, and with probability η flips its state independently.
• The order parameter Ψ(t) = ⟨σi(t)⟩ is defined as the average magnetization, and its time evolution is modeled via a Fokker-Planck-type equation derived from the master equation of the stochastic process.
• The fixed points of the order parameter dynamics are computed analytically by solving a nonlinear integral equation, leading to a bifurcation diagram that reveals the phase transition.
• The critical noise level ηc is obtained by solving the fixed-point equation, and the critical exponent is extracted from the scaling behavior of Ψ near ηc.
• The results are validated numerically by simulating two network realizations: one with constant weights (cij = 1) and another with uniformly distributed weights (cij ∈ [0,1]).

Experimental results

Research questions

• RQ1 Does a critical noise level ηc exist below which the neural network remains dynamically organized and above which it becomes disordered?
• RQ2 What is the nature of the phase transition—first-order or second-order—and what is the associated critical exponent?
• RQ3 How does the critical noise level ηc depend on the network's connectivity K and the distribution of connection weights?
• RQ4 To what extent is the phase transition robust to changes in the weight distribution, particularly for non-symmetric distributions?
• RQ5 Can the order parameter dynamics be exactly solved analytically under general conditions on the network's quenched connectivity and weights?

Key findings

• The system undergoes a second-order dynamical phase transition at a critical noise level ηc, with a critical exponent of 1/2, as confirmed by the scaling behavior of the order parameter Ψ near ηc.
• For the case of constant weights (cij = 1), the critical noise level is ηc ≈ 0.3153, and the transition is well described by the analytical bifurcation curve.
• For uniformly distributed weights (cij ∈ [0,1]), the critical noise level is ηc ≈ 0.2838, and the analytical solution matches numerical simulations with high precision.
• The critical noise level ηc increases with connectivity K and asymptotically approaches 1/2 as K → ∞, indicating that higher connectivity enhances robustness to noise.
• For K ≤ 2, ηc = 0, meaning the system becomes disordered even at arbitrarily small noise, and no phase transition occurs.
• The analytical framework holds under very general conditions: the only requirement is that the weight distribution P_c(x) is non-symmetric, and the results are robust to the specific form of P_c(x).