[Paper Review] Algorithmic pure states for the negative spherical perceptron

A. El Alaoui, Mark Sellke|arXiv (Cornell University)|Oct 29, 2020

Caveolin-1 and cellular processes4 citations

TL;DR

This paper presents an efficient algorithm that, given oracle access to the solution of the Parisi variational principle, computes a vector $σ\hat{}$ lying on a sphere of radius $\sqrt{\bar{q}N}$ with $\bar{q} \in (0,1)$, satisfying the spherical perceptron constraints $\langle \bm{g}_a, \hat{\bm{\sigma}} \rangle \geq \kappa\sqrt{N}$ for all $a \leq M$ when $\kappa < 0$. The algorithm exploits the conjectured full replica-symmetry breaking (FRSB) structure near the satisfiability threshold, and $\bar{q} \to 1$ as $\alpha \to \alpha_{\text{SAT}}(\kappa)$, implying near-optimal alignment with the constraint hyperplanes.

ABSTRACT

We consider the spherical perceptron with Gaussian disorder. This is the set $S$ of points $σ\in \mathbb{R}^N$ on the sphere of radius $\sqrt{N}$ satisfying $\langle g_a , σ angle \ge κ\sqrt{N}\,$ for all $1 \le a \le M$, where $(g_a)_{a=1}^M$ are independent standard gaussian vectors and $κ\in \mathbb{R}$ is fixed. Various characteristics of $S$ such as its surface measure and the largest $M$ for which it is non-empty, were computed heuristically in statistical physics in the asymptotic regime $N o \infty$, $M/N o α$. The case $κ&lt;0$ is of special interest as $S$ is conjectured to exhibit a hierarchical tree-like geometry known as "full replica-symmetry breaking" (FRSB) close to the satisfiability threshold $α_{ ext{SAT}}(κ)$, and whose characteristics are captured by a Parisi variational principle akin to the one appearing in the Sherrington-Kirkpatrick model. In this paper we design an efficient algorithm which, given oracle access to the solution of the Parisi variational principle, exploits this conjectured FRSB structure for $κ&lt;0$ and outputs a vector $\hatσ$ satisfying $\langle g_a , \hatσ angle \ge κ\sqrt{N}$ for all $1\le a \le M$ and lying on a sphere of non-trivial radius $\sqrt{\bar{q} N}$, where $\bar{q} \in (0,1)$ is the right-end of the support of the associated Parisi measure. We expect $\hatσ$ to be approximately the barycenter of a pure state of the spherical perceptron. Moreover we expect that $\bar{q} o 1$ as $α o α_{ ext{SAT}}(κ)$, so that $\big\langle g_a,\hatσ/|\hatσ|\big angle \geq (κ-o(1))\sqrt{N}$ near criticality.

Motivation & Objective

• To design an efficient algorithm that finds a solution to the spherical perceptron problem with negative $\kappa$, exploiting the conjectured full replica-symmetry breaking (FRSB) geometry.
• To construct a vector $\hat{\bm{\sigma}}$ lying on a sphere of non-trivial radius $\sqrt{\bar{q}N}$ with $\bar{q} \in (0,1)$, satisfying all $M$ constraints $\langle \bm{g}_a, \hat{\bm{\sigma}} \rangle \geq \kappa\sqrt{N}$.
• To provide a constructive method for identifying a vector that approximates the barycenter of a pure state in the FRSB regime, expected to be near the solution space's core.
• To establish that $\bar{q} \to 1$ as $\alpha \to \alpha_{\text{SAT}}(\kappa)$, implying near-optimal performance near the satisfiability threshold.

Proposed method

• The algorithm uses oracle access to the solution of the Parisi variational principle, which governs the FRSB structure in the negative $\kappa$ regime.
• It constructs $\hat{\bm{\sigma}}$ via a pathwise stochastic process derived from a backward Kolmogorov equation with time-dependent diffusion coefficient $\gamma(t)$, modeling the FRSB dynamics.
• The method relies on solving a system of SDEs driven by Brownian motion, with drift determined by the derivative of the Parisi-type potential $\Phi_\gamma$.
• It employs Itô's formula and Gronwall-type estimates to bound higher-order derivatives of $\Phi_\gamma$, ensuring regularity and existence of solutions.
• The algorithm leverages the fact that $\partial_x^2 \Phi_\gamma$ is bounded under monotonicity of the Parisi measure, enabling stability in the iterative construction.
• It uses a discrete-time approximation to the SDE to derive bounds on the second derivative of $\Phi_\gamma$, ensuring the solution remains well-behaved.

Experimental results

Research questions

• RQ1 Can an efficient algorithm be designed to find a solution to the negative spherical perceptron problem when the solution space exhibits full replica-symmetry breaking (FRSB) geometry?
• RQ2 What is the geometric role of the parameter $\bar{q} \in (0,1)$, and how does it relate to the barycenter of a pure state in the FRSB regime?
• RQ3 How does the radius $\sqrt{\bar{q}N}$ of the solution vector $\hat{\bm{\sigma}}$ behave as the system approaches the satisfiability threshold $\alpha_{\text{SAT}}(\kappa)$?
• RQ4 Can the Parisi variational principle be algorithmically leveraged to construct a solution vector that satisfies all constraints while lying on a sphere of non-trivial radius?
• RQ5 What is the relationship between the solution $\hat{\bm{\sigma}}$ and the limiting behavior of the surface measure of the solution set $\mathrm{S}_{M,N}(\bm{G})$ near criticality?

Key findings

• The algorithm successfully computes a vector $\hat{\bm{\sigma}}$ satisfying $\langle \bm{g}_a, \hat{\bm{\sigma}} \rangle \geq \kappa\sqrt{N}$ for all $1 \leq a \leq M$, with $\|\hat{\bm{\sigma}}\|_2^2 = \bar{q}N$ and $\bar{q} \in (0,1)$, under the assumption of oracle access to the Parisi solution.
• The value $\bar{q}$ corresponds to the right endpoint of the support of the Parisi measure associated with the FRSB solution, and is determined by the variational principle.
• As $\alpha \to \alpha_{\text{SAT}}(\kappa)$, the algorithm's output satisfies $\bar{q} \to 1$, implying $\left\langle \bm{g}_a, \frac{\hat{\bm{\sigma}}}{|\hat{\bm{\sigma}}|} \right\rangle \geq (\kappa - o(1))\sqrt{N}$, approaching the optimal constraint bound.
• The solution $\hat{\bm{\sigma}}$ is expected to approximate the barycenter of a pure state in the FRSB geometry, consistent with statistical physics predictions.
• The method establishes the boundedness of $\partial_x^2 \Phi_\gamma$ via discrete-time approximation and Gronwall estimates, ensuring the regularity of the solution to the associated PDE system.
• The algorithm’s design is robust under the assumption that the Parisi variational principle is solvable, and it provides a constructive pathway to near-optimal solutions in the FRSB regime.