[Paper Review] Substochastic Monte Carlo Algorithms

Michael Jarret, Brad Lackey|arXiv (Cornell University)|Apr 28, 2017

Computability, Logic, AI Algorithms10 references3 citations

TL;DR

This paper introduces Substochastic Monte Carlo (SSMC), a general classical optimization algorithm that simulates a population of walkers evolving under substochastic dynamics to solve combinatorial optimization problems. SSMC is shown to be exponentially faster than Go-With-The-Winners (GWW) and even outperform the quantum annealing algorithm that inspired it, while also enabling gradient descent against biases in general search spaces.

ABSTRACT

In this paper we introduce and formalize Substochastic Monte Carlo (SSMC) algorithms. These algorithms, originally intended to be a better classical foil to quantum annealing than simulated annealing, prove to be worthy optimization algorithms in their own right. In SSMC, a population of walkers is initialized according to a known distribution on an arbitrary search space and varied into the solution of some optimization problem of interest. The first argument of this paper shows how an existing classical algorithm, "Go-With-The-Winners" (GWW), is a limiting case of SSMC when restricted to binary search and particular driving dynamics. Although limiting to GWW, SSMC is more general. We show that (1) GWW can be efficiently simulated within the SSMC framework, (2) SSMC can be exponentially faster than GWW, (3) by naturally incorporating structural information, SSMC can exponentially outperform the quantum algorithm that first inspired it, and (4) SSMC exhibits desirable search features in general spaces. Our approach combines ideas from genetic algorithms (GWW), theoretical probability (Fleming-Viot processes), and quantum computing. Not only do we demonstrate that SSMC is often more efficient than competing algorithms, but we also hope that our results connecting these disciplines will impact each independently. An implemented version of SSMC has previously enjoyed some success as a competitive optimization algorithm for Max-$k$-SAT.

Motivation & Objective

• To formalize Substochastic Monte Carlo (SSMC) as a general-purpose classical optimization algorithm beyond the scope of binary or layered graphs.
• To establish a rigorous connection between SSMC and Go-With-The-Winners (GWW), showing that GWW is a limiting case of SSMC under specific dynamics.
• To demonstrate that SSMC can achieve exponential speedups over GWW and the quantum annealing algorithm that inspired it, even with identical annealing schedules.
• To show that SSMC naturally incorporates structural information in search spaces, enabling efficient exploration of complex landscapes including those with local optima and barriers.
• To validate SSMC's ability to perform gradient descent against biases in general graphs, not restricted to tree-like or layered structures.

Proposed method

• SSMC models a population of walkers evolving via a continuous-time substochastic process governed by a time-dependent transition rate matrix $ H(t) = a(t)L + b(t)W $, where $ L $ is a graph Laplacian and $ W $ is a diagonal matrix encoding objective function values.
• The process is renormalized through a death-and-repopulation mechanism: when a walker is absorbed (dies), it is replaced by a copy of a surviving walker, preserving population size.
• The dynamics follow the master equation $ \frac{d\psi}{dt}(t;y) = -\sum_x H(t)_{y,x} \psi(t;x) $, with initial distribution $ \psi_0 $, evolving toward a solution encoded in $ \psi(T) $.
• The algorithm uses a linear annealing schedule with $ a(t) = 1 - t/T $, $ b(t) = t/T $, interpolating between a driving dynamics (via $ L $) and the optimization objective (via $ W $).
• SSMC leverages the Fleming-Viot process framework to simulate a non-conservative, substochastic diffusion that concentrates walkers on low-energy states.
• The method enables controlled descent by tuning the time parameter $ t $, allowing trade-offs between random exploration and directed search, especially effective near $ t \to 1 $.

Experimental results

Research questions

• RQ1 Can Substochastic Monte Carlo (SSMC) be formally established as a generalization of Go-With-The-Winners (GWW), and under what conditions does it converge to GWW?
• RQ2 Does SSMC achieve exponential speedups over GWW in optimization tasks, particularly in structured search spaces?
• RQ3 Can SSMC outperform the quantum annealing algorithm that inspired it, even when using the same annealing schedule and without quantum resources?
• RQ4 How does SSMC perform in general search spaces—especially in terms of gradient descent against biases and barrier crossing—beyond the constraints of layered or binary graphs?
• RQ5 What mechanisms allow SSMC to efficiently cross high barriers in the energy landscape, and how does the death-and-repopulation process enable exponential growth across such barriers?

Key findings

• SSMC is exponentially faster than Go-With-The-Winners (GWW) under the same driving dynamics, with the speedup arising from the ability to exploit structural information in the search space.
• SSMC can be exponentially faster than the quantum annealing (QA) algorithm that inspired it, even when using identical annealing schedules, due to more effective population dynamics and bias exploitation.
• GWW is formally shown to be a limiting case of SSMC when restricted to binary search spaces and specific driving dynamics, confirming SSMC's generality.
• SSMC enables gradient descent against biases in general search spaces, with the probability of transitioning to lower-energy states approaching 1 as $ t \to 1 $, under appropriate conditions.
• The algorithm exhibits a controlled trade-off between exploration and exploitation, with the ability to delay descent until after local optima are identified, enhancing global search capability.
• SSMC can cross high barriers with bounded probability through the death-and-repopulation mechanism, enabling exponential growth of walkers across barriers, as demonstrated in combinatorial structures like combs.