[Paper Review] Self-learning Machines based on Hamiltonian Echo Backpropagation

Víctor López-Pastor, Florian Marquardt|Figshare|Mar 8, 2021

Neural Networks and Applications80 references4 citations

TL;DR

This paper introduces Hamiltonian Echo Backpropagation (HEB), a general method for training self-learning machines in time-reversible Hamiltonian systems by leveraging time-reversal symmetry and injecting a small error signal to induce autonomous gradient updates. The method enables efficient, fully physical backpropagation without external feedback or digital processing, demonstrating robust training in coupled nonlinear wave fields and other systems.

ABSTRACT

A physical self-learning machine can be defined as a nonlinear dynamical system that can be trained on data (similar to artificial neural networks), but where the update of the internal degrees of freedom that serve as learnable parameters happens autonomously. In this way, neither external processing and feedback nor knowledge of (and control of) these internal degrees of freedom is required. We introduce a general scheme for self-learning in any time-reversible Hamiltonian system. We illustrate the training of such a self-learning machine numerically for the case of coupled nonlinear wave fields.

Motivation & Objective

• To develop a general, physically realizable training method for self-learning machines that operate without external feedback or digital processing.
• To overcome limitations of existing optical and neuromorphic learning systems by enabling gradient descent via intrinsic physical dynamics.
• To demonstrate that any time-reversible Hamiltonian system can be trained using HEB, regardless of specific nonlinearities or system details.
• To enable stable parameter learning by storing parameters in degenerate ground states of symmetric Hamiltonians, with robustness to small perturbations.
• To extend applicability to both continuous and discrete learning parameters, including analogies to thermal annealing for discrete optimization.

Proposed method

• The method relies on time-reversal symmetry: a forward pass is followed by a time-reversed echo of the system's dynamics.
• A small error signal is injected during the echo phase to break time-reversal symmetry and generate a physical gradient signal.
• The resulting dynamics naturally update the learnable parameters via a physical backpropagation mechanism, mimicking stochastic gradient descent.
• The approach is general and does not require engineering specific nonlinear elements or precise control of the system's internal parameters.
• The learning parameter is encoded in a continuous or discrete symmetry of the Hamiltonian, such as phase or amplitude differences in parametrically driven oscillators.
• Stability is enhanced by storing parameters in a degenerate ground state manifold, where small perturbations do not destabilize the learned configuration.

Figure 1: Different types of physical learning machines. (a) Physical learning machine requiring feedback based on the outcome, to tune the internal "learning" parameters $\theta$ inside the physical device. (b) A self-learning machine does not involve feedback. It updates the learning parameters in

Experimental results

Research questions

• RQ1 Can a self-learning machine be trained using only intrinsic physical dynamics, without external feedback or digital computation?
• RQ2 How can gradient descent be physically realized in a time-reversible Hamiltonian system?
• RQ3 What role does time-reversal symmetry play in enabling autonomous parameter updates?
• RQ4 How can learning parameters be stored stably in physical systems with inherent symmetries?
• RQ5 Can the method be applied to both continuous and discrete parameter spaces, and how does it relate to thermal annealing?

Key findings

• Hamiltonian Echo Backpropagation enables autonomous training of self-learning machines in any time-reversible Hamiltonian system, without requiring knowledge of or control over internal parameters.
• The method achieves physical gradient descent by injecting a small error signal during a time-reversed echo, inducing parameter updates through the system's natural dynamics.
• Numerical simulations demonstrate successful training in coupled nonlinear wave fields, confirming the method's feasibility and efficiency.
• Stable parameter storage is achieved when learning parameters are encoded in degenerate ground states of symmetric Hamiltonians, which are robust to small perturbations.
• The method can be adapted to discrete parameters by interpreting the learning process as thermal annealing, where the system converges to local minima of an effective Hamiltonian.
• The approach is general and does not depend on specific system details, such as nonlinearities or geometric arrangements, making it applicable to diverse physical platforms including optics and parametric oscillators.

Figure 2: Overview: The landscape of physical learning machines. Our work falls into the broad class of optimization approaches. It belongs to the small class of works that involve both physical backpropagation and self-learning, but in contrast to other approaches our scheme is universal, i.e. inde