[Paper Review] Noise-injected analog Ising machines enable ultrafast statistical sampling and machine learning

Fabian Böhm, Diego Alonso-Urquijo+2 more|arXiv (Cornell University)|Dec 21, 2021

Advanced Memory and Neural ComputingEngineering55 references40 citations

TL;DR

This paper proposes noise injection as a universal method to enable ultrafast Boltzmann sampling in analog Ising machines, overcoming their inherent inability to reach thermal equilibrium. By injecting broadband noise into a time-multiplexed opto-electronic Ising machine, the authors demonstrate accurate sampling of Boltzmann distributions and unsupervised training of restricted Boltzmann machines with performance matching software-based methods, achieving sampling rates up to GSamples/s—orders of magnitude faster than conventional approaches.

ABSTRACT

Ising machines are a promising non-von-Neumann computational concept for neural network training and combinatorial optimization. However, while various neural networks can be implemented with Ising machines, their inability to perform fast statistical sampling makes them inefficient for training neural networks compared to digital computers. Here, we introduce a universal concept to achieve ultrafast statistical sampling with analog Ising machines by injecting noise. With an opto-electronic Ising machine, we experimentally demonstrate that this can be used for accurate sampling of Boltzmann distributions and for unsupervised training of neural networks, with equal accuracy as software-based training. Through simulations, we find that Ising machines can perform statistical sampling orders-of-magnitudes faster than software-based methods. This enables the use of Ising machines beyond combinatorial optimization and makes them into efficient tools for machine learning and other applications.

Motivation & Objective

• To address the critical limitation of analog Ising machines in performing fast and accurate Boltzmann sampling, which hinders their use in machine learning.
• To develop a universal, hardware-agnostic method for inducing thermal equilibrium in analog Ising systems without complex temperature control.
• To demonstrate that noise-injected Ising machines can achieve unsupervised training accuracy comparable to digital software-based training.
• To quantify the sampling speed advantage of analog Ising machines over software-based methods using numerical simulations.
• To validate the method across multiple types of gain-dissipative systems, proving its universality.

Proposed method

• Injecting broadband Gaussian white noise into a time-multiplexed opto-electronic Ising machine to drive the system into thermal equilibrium at controllable effective temperatures.
• Using a feedback loop with a field-programmable gate array (FPGA) to dynamically control coupling weights and biases, enabling real-time reconfiguration of the Ising Hamiltonian.
• Mapping binary Ising spins to continuous analog amplitudes via a pitchfork bifurcation in nonlinear optical systems, where spin states are determined by the sign of the amplitude.
• Employing the Metropolis-Hastings algorithm as a benchmark for evaluating sampling accuracy, comparing it against samples generated by the noise-injected Ising machine.
• Simulating various gain-dissipative systems (polynomial, clipped, sigmoid nonlinearities) to test the universality of the noise-injection approach across different physical implementations.
• Using pseudolikelihood and prediction accuracy as metrics to evaluate unsupervised training performance on MNIST digit recognition.

Experimental results

Research questions

• RQ1 Can noise injection induce accurate Boltzmann sampling in analog Ising machines that are otherwise limited to ground-state computation?
• RQ2 What is the relationship between injected noise variance and the effective temperature of the Ising system?
• RQ3 Can noise-injected Ising machines achieve unsupervised training accuracy comparable to software-based methods in a real-world machine learning task?
• RQ4 How does the sampling speed of noise-injected Ising machines compare to software-based sampling methods?
• RQ5 Is the noise-injection method universally applicable across different types of gain-dissipative physical systems?

Key findings

• The noise-injected opto-electronic Ising machine achieved Boltzmann sampling accuracy matching that of the Metropolis-Hastings algorithm across a wide temperature range, including near the critical phase transition point.
• The effective temperature of the Ising system was linearly proportional to the variance of the injected noise, enabling precise control over the sampling distribution.
• Unsupervised training of a restricted Boltzmann machine using noise-injected sampling achieved a test prediction accuracy of 93.8% (polynomial), 95.3% (clipped), and 94.4% (sigmoid), matching the performance of software-based training.
• Numerical simulations projected that spatially multiplexed analog Ising machines could achieve sampling rates of up to GSamples/s using off-the-shelf components, representing a speedup of several orders of magnitude over software-based methods.
• The noise-injection method was successfully validated across three distinct types of gain-dissipative systems—polynomial, clipped, and sigmoid nonlinearities—demonstrating its universality.
• The method enables analog Ising machines to transition from being limited to combinatorial optimization to becoming viable platforms for general-purpose machine learning and statistical sampling.