[Paper Review] Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation

Benjamin Scellier, Yoshua Bengio|arXiv (Cornell University)|Feb 16, 2016

Advanced Memory and Neural Computing30 references15 citations

TL;DR

This paper introduces Equilibrium Propagation, a biologically plausible learning framework for energy-based models that computes gradients using a single type of neural computation in both inference and error-backpropagation phases. By nudging output units toward targets during a second phase, the method implicitly back-propagates error derivatives through recurrent dynamics, achieving gradient descent on a well-defined objective without requiring separate backpropagation circuits or assumptions like symmetric weights.

ABSTRACT

We introduce Equilibrium Propagation, a learning framework for energy-based models. It involves only one kind of neural computation, performed in both the first phase (when the prediction is made) and the second phase of training (after the target or prediction error is revealed). Although this algorithm computes the gradient of an objective function just like Backpropagation, it does not need a special computation or circuit for the second phase, where errors are implicitly propagated. Equilibrium Propagation shares similarities with Contrastive Hebbian Learning and Contrastive Divergence while solving the theoretical issues of both algorithms: our algorithm computes the gradient of a well defined objective function. Because the objective function is defined in terms of local perturbations, the second phase of Equilibrium Propagation corresponds to only nudging the prediction (fixed point, or stationary distribution) towards a configuration that reduces prediction error. In the case of a recurrent multi-layer supervised network, the output units are slightly nudged towards their target in the second phase, and the perturbation introduced at the output layer propagates backward in the hidden layers. We show that the signal 'back-propagated' during this second phase corresponds to the propagation of error derivatives and encodes the gradient of the objective function, when the synaptic update corresponds to a standard form of spike-timing dependent plasticity. This work makes it more plausible that a mechanism similar to Backpropagation could be implemented by brains, since leaky integrator neural computation performs both inference and error back-propagation in our model. The only local difference between the two phases is whether synaptic changes are allowed or not.

Motivation & Objective

• To develop a biologically plausible alternative to backpropagation that avoids the need for separate error-propagation circuits.
• To resolve theoretical issues in contrastive learning methods like Contrastive Hebbian Learning and Contrastive Divergence.
• To show that error derivatives can be implicitly back-propagated through recurrent dynamics using only local perturbations.
• To establish a connection between equilibrium dynamics and gradient descent via spike-timing dependent plasticity rules.
• To demonstrate training feasibility of deep recurrent networks on MNIST using this framework.

Proposed method

• Uses a single recurrent neural network with leaky integrator dynamics for both inference and error propagation.
• Performs two phases: first, the network relaxes to a fixed point under input clamping (inference); second, output units are slightly nudged toward targets (training).
• The second-phase dynamics implicitly propagate error derivatives backward through hidden layers via synaptic weight updates.
• Synaptic updates follow a contrastive Hebbian rule that increases energy of the first-phase fixed point and decreases it for the second-phase fixed point.
• Leverages results from Bengio et al. (2017) to show that early neural activity changes in the second phase correspond to error derivatives.
• Employs a learning rule derived from the gradient of a squared-error objective function, with weight updates based on temporal derivatives of postsynaptic activity.

Experimental results

Research questions

• RQ1 Can a single neural computation mechanism perform both inference and error back-propagation in a biologically plausible way?
• RQ2 Does implicit error back-propagation via small perturbations yield the correct gradient for training energy-based models?
• RQ3 How does the second-phase dynamics relate to the true gradient of the loss function in recurrent networks?
• RQ4 Can Equilibrium Propagation train deep, multi-layered recurrent networks on real-world tasks like MNIST?
• RQ5 Is the synaptic update rule equivalent to a form of spike-timing dependent plasticity (STDP) under the proposed dynamics?

Key findings

• Equilibrium Propagation computes the gradient of a well-defined objective function (squared error) without requiring separate backpropagation circuits.
• The second-phase dynamics implicitly back-propagate error derivatives through hidden layers, matching the true gradient of the loss.
• The synaptic update rule corresponds to a form of spike-timing dependent plasticity, making it biologically plausible.
• Multi-layer recurrent networks with 1, 2, and 3 hidden layers successfully trained on the permutation-invariant MNIST task using this framework.
• The method avoids theoretical flaws of prior contrastive learning algorithms, such as cycling in CD-1 or mode collapse in contrastive Hebbian learning.
• Hyperparameters including relaxation iterations (20–500), perturbation strength (ε=0.5), and learning rates (αk) were tuned to achieve convergence on MNIST.