[Paper Review] Agnostic Physics-Driven Deep Learning

Benjamin Scellier, Siddhartha Mishra+2 more|arXiv (Cornell University)|May 30, 2022

Neural Networks and Applications4 citations

TL;DR

This paper introduces Agnostic Equilibrium Propagation (Æqprop), a physics-driven learning framework that enables gradient descent in physical systems without requiring explicit gradient computation or knowledge of the system's analytical energy function. By treating model parameters as dynamic variables that minimize system energy and using nudging to induce parameter updates, Æqprop achieves true first-order optimization via physical dynamics, enabling efficient, hardware-agnostic machine learning in biological and analog systems.

ABSTRACT

This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging towards the correct response. In Aeqprop, the specifics of the system do not have to be known: the procedure is based only on external manipulations, and produces a stochastic gradient descent without explicit gradient computations. Thanks to nudging, the system performs a true, order-one gradient step for each training sample, in contrast with order-zero methods like reinforcement or evolutionary strategies, which rely on trial and error. This procedure considerably widens the range of potential hardware for statistical learning to any system with enough controllable parameters, even if the details of the system are poorly known. Aeqprop also establishes that in natural (bio)physical systems, genuine gradient-based statistical learning may result from generic, relatively simple mechanisms, without backpropagation and its requirement for analytic knowledge of partial derivatives.

Motivation & Objective

• To overcome the limitations of existing physical learning methods that require explicit knowledge of energy function derivatives and parameter update mechanisms.
• To enable gradient-based learning in physical systems where the internal dynamics are poorly understood or unknown.
• To eliminate the need for storing intermediate equilibrium states during training, a major bottleneck in physical implementations.
• To demonstrate that first-order optimization—equivalent to stochastic gradient descent—can emerge from simple physical mechanisms like energy minimization and nudging.
• To establish a framework where learning is inherently hardware-agnostic, applicable to any controllable physical system with sufficient degrees of freedom.

Proposed method

• Treats trainable parameters as floating variables that minimize the system's energy function, just like state variables.
• Uses a control knob to maintain parameters at their current value during the first phase, ensuring stable initialization.
• Applies a small perturbation (nudging) to the output energy function in the second phase to guide the system toward a desired response.
• Derives parameter updates from the change in equilibrium state under nudging, showing that this change corresponds to a true gradient step.
• Employs a Lyapunov function to prove convergence and stability, with theoretical guarantees on loss reduction even with large step sizes.
• Relies only on external manipulations (input, control knob, nudging) and does not require explicit gradient computation or analytical knowledge of partial derivatives.

Experimental results

Research questions

• RQ1 Can gradient descent be achieved in physical systems without explicit gradient computation or knowledge of the energy function’s analytical form?
• RQ2 Can parameter updates in physical systems be induced directly by physical dynamics, without relying on external digital control or non-volatile memory?
• RQ3 Does the proposed method provide stronger convergence guarantees than standard stochastic gradient descent, especially under large step sizes?
• RQ4 Can the learning process be fully embedded in the physical laws of the system, enabling hardware-agnostic machine learning?
• RQ5 Is it possible to achieve first-order optimization in physical systems using only energy minimization and nudging, without backpropagation or explicit derivative computation?

Key findings

• Æqprop achieves one full step of stochastic gradient descent per training sample through physical dynamics, without explicit gradient computation.
• The method is agnostic to the analytical form of the energy function, requiring only external control and observation of system states.
• Parameter updates in Æqprop correspond exactly to a true gradient step with respect to the loss function, as proven in Theorem 1.
• The Pessimistic variant of Æqprop guarantees a reduction in a tight bound on the loss function at each step, even with large learning rates.
• The method eliminates the need to store the first equilibrium state, overcoming a key limitation of previous Eqprop-based approaches.
• Æqprop enables end-to-end physical learning in systems where parameters are not pre-defined or stored, such as in analog or biological hardware.