[论文解读] Algorithmic Thresholds in Mean Field Spin Glasses

A. El Alaoui, Andrea Montanari|arXiv (Cornell University)|Sep 24, 2020

Theoretical and Computational Physics参考文献 39被引用 17

一句话总结

本文首次实现了用于优化平均场自旋玻璃的近似消息传递（AMP）算法，并通过数值实验验证了其性能，该算法利用近优态的超度量树结构。在Sherrington-Kirkpatrick（SK）模型中，该算法渐近地实现了任意良好的近似；在3自旋模型中，实现了与理论预测一致的常数近似比，优于Glauber动力学。

ABSTRACT

Optimizing a high-dimensional non-convex function is, in general, computationally hard and many problems of this type are hard to solve even approximately. Complexity theory characterizes the optimal approximation ratios achievable in polynomial time in the worst case. On the other hand, when the objective function is random, worst case approximation ratios are overly pessimistic. Mean field spin glasses are canonical families of random energy functions over the discrete hypercube $\{-1,+1\}^N$. The near-optima of these energy landscapes are organized according to an ultrametric tree-like structure, which enjoys a high degree of universality. Recently, a precise connection has begun to emerge between this ultrametric structure and the optimal approximation ratio achievable in polynomial time in the typical case. A new approximate message passing (AMP) algorithm has been proposed that leverages this connection. The asymptotic behavior of this algorithm has been analyzed, conditional on the nature of the solution of a certain variational problem. In this paper we describe the first implementation of this algorithm and the first numerical solution of the associated variational problem. We test our approach on two prototypical mean-field spin glasses: the Sherrington-Kirkpatrick (SK) model, and the $3$-spin Ising spin glass. We observe that the algorithm works well already at moderate sizes ($N\gtrsim 1000$) and its behavior is consistent with theoretical expectations. For the SK model it asymptotically achieves arbitrarily good approximations of the global optimum. For the $3$-spin model, it achieves a constant approximation ratio that is predicted by the theory, and it appears to beat the `threshold energy' achieved by Glauber dynamics. Finally, we observe numerically that the intermediate states generated by the algorithm have the properties of ancestor states in the ultrametric tree.

研究动机与目标

• 实现并数值验证一种用于优化平均场自旋玻璃中高维随机能量函数的近似消息传递（AMP）算法。
• 在多项式时间内求解算法阈值背后的变分问题，从而实现对近优配置的实际计算。
• 检验AMP算法是否实现了基于自旋玻璃模型中近优态超度量结构所预测的理论近似比。
• 通过数值方法验证算法生成的中间态是否对应于超度量树结构中的祖先态。
• 证明该算法在能量近似方面优于标准的马尔可夫链蒙特卡洛方法（如Glauber动力学）。

提出的方法

• 该算法基于广义TAP方程的迭代近似消息传递框架，其更新规则由一个表征超度量树结构的函数γ(s)所定义的变分原理导出。
• 该方法引入步长δ，并随时间t ∈ [0,1]演化磁化向量m^t，以近似求解TAP自由能最小化问题。
• 关键组成部分包括广义TAP自由能F_TAP(m) = H_N(m) + ∑ᵢ Λ_γ(t, mᵢ) − (N/2)∫ₜ¹ sξ''(s)γ(s)ds，该自由能在算法路径上被最小化。
• 该算法强制TAP方程的平稳性：m = F(m; t)，其中F通过Φ_γ(t; z)的Legendre变换的逆定义。
• 在N ≥ 1000的SK（2自旋）和3自旋伊辛模型上进行数值实验，通过独立实现评估收敛性和误差标度。
• 通过监测TAP自由能和TAP方程误差Δ_t(δ,N)，验证理论一致性，其在N→∞、δ→0极限下呈现Θ(δ)的标度。

实验结果

研究问题

• RQ1 AMP算法在实际中能否实现平均场自旋玻璃的理论算法阈值？
• RQ2 AMP算法的中间态演化是否对应于近优态超度量树结构中的层级祖先态？
• RQ3 AMP算法在SK模型和3自旋模型中实现的实证近似比是多少？与Glauber动力学相比如何？
• RQ4 满足广义TAP方程的误差如何随步长δ和系统尺寸N变化？
• RQ5 广义TAP自由能在算法执行过程中是否近似恒定？这是否与理论预测一致？

主要发现

• 在Sherrington-Kirkpatrick（SK）模型中，随着N增大，AMP算法实现了对全局最优解的任意良好近似，与理论预测一致。
• 在3自旋伊辛模型中，该算法实现了与变分原理所预测的理论阈值一致的常数近似比。
• 该算法优于Glauber动力学，在能量近似方面表现更优，达到的能量低于后者的“阈值能量”。
• 数值结果表明，算法生成的中间态表现出超度量树结构中祖先态的结构性质，包括层级重叠结构。
• 广义TAP自由能在算法执行过程中保持近似恒定，其偏差按Θ(δ)标度变化，与大N极限下的理论预期一致。
• 在每次迭代中，TAP方程近似满足，误差按O(δ)减小，且在独立实现之间误差高度集中。