[Paper Review] Biologically-plausible backpropagation through arbitrary timespans via local neuromodulators

Yuhan Helena Liu, Stephen M. Smith+3 more|arXiv (Cornell University)|Jun 2, 2022

Advanced Memory and Neural Computing5 citations

TL;DR

This paper proposes ModProp, a biologically plausible learning rule that enables backpropagation through arbitrary timespans in recurrent neural networks using local neuromodulators like neuropeptides. By leveraging diffusive, cell-type-specific modulatory signals to convolve eligibility traces via time-invariant filters, ModProp achieves superior performance on temporal tasks compared to truncated gradient methods, offering a causal, low-complexity alternative to backpropagation through time.

ABSTRACT

The spectacular successes of recurrent neural network models where key parameters are adjusted via backpropagation-based gradient descent have inspired much thought as to how biological neuronal networks might solve the corresponding synaptic credit assignment problem. There is so far little agreement, however, as to how biological networks could implement the necessary backpropagation through time, given widely recognized constraints of biological synaptic network signaling architectures. Here, we propose that extra-synaptic diffusion of local neuromodulators such as neuropeptides may afford an effective mode of backpropagation lying within the bounds of biological plausibility. Going beyond existing temporal truncation-based gradient approximations, our approximate gradient-based update rule, ModProp, propagates credit information through arbitrary time steps. ModProp suggests that modulatory signals can act on receiving cells by convolving their eligibility traces via causal, time-invariant and synapse-type-specific filter taps. Our mathematical analysis of ModProp learning, together with simulation results on benchmark temporal tasks, demonstrate the advantage of ModProp over existing biologically-plausible temporal credit assignment rules. These results suggest a potential neuronal mechanism for signaling credit information related to recurrent interactions over a longer time horizon. Finally, we derive an in-silico implementation of ModProp that could serve as a low-complexity and causal alternative to backpropagation through time.

Motivation & Objective

• To address the synaptic credit assignment problem in biological neural circuits, particularly for long-term recurrent dependencies.
• To develop a biologically plausible mechanism for propagating credit signals over arbitrary time horizons, beyond the limitations of current temporal truncation-based approximations.
• To explore the role of local, diffusive neuromodulators—especially neuropeptides—in enabling long-range temporal credit assignment.
• To propose a computationally efficient, causal learning rule that mimics backpropagation through time without requiring non-local or non-causal information.
• To validate the method on benchmark temporal tasks and compare its performance to existing biologically plausible learning rules.

Proposed method

• ModProp introduces a gradient update rule that uses local, diffusive neuromodulators to propagate credit information through time via a convolution of eligibility traces with time-invariant, synapse-type-specific filter taps.
• The method models the action of neuromodulators as causal, time-invariant filters that act on postsynaptic eligibility traces, enabling long-horizon credit assignment without backpropagating error signals directly.
• The learning rule is derived from a mathematical analysis of backpropagation through time, approximated using a causal, local, and biologically plausible mechanism involving GPCR-activated modulatory signals.
• Eligibility traces are maintained at synapses based on pre- and postsynaptic spike timing, and modulated by slow, diffusible signals that reflect reward or error signals over extended durations.
• The update rule is implemented in an in-silico model using standard training protocols (e.g., Adam optimizer) and validated across multiple temporal tasks, including delayed XOR, pattern generation, copy, and MNIST tasks.
• Hyperparameters such as the neuromodulator decay rate (μ) are tuned via grid search to optimize performance on each task.

Experimental results

Research questions

• RQ1 Can local, diffusive neuromodulators such as neuropeptides serve as a biologically plausible mechanism for propagating credit signals over arbitrary timespans in recurrent networks?
• RQ2 How can a causal, local, and synapse-specific learning rule approximate backpropagation through time without requiring non-local error signals?
• RQ3 To what extent does the proposed ModProp rule outperform existing biologically plausible temporal credit assignment methods on benchmark temporal tasks?
• RQ4 What is the role of cell-type-specific, GPCR-mediated neuromodulation in shaping long-term synaptic plasticity and credit assignment?
• RQ5 Can a low-complexity, causal learning rule achieve performance comparable to exact backpropagation through time in recurrent networks?

Key findings

• ModProp outperforms existing biologically plausible learning rules—such as those based on temporal truncation—on all benchmark temporal tasks, including delayed XOR, pattern generation, and copy tasks.
• In the delayed XOR task, ModProp achieved a test accuracy of 98.5%, significantly outperforming the next-best method (92.1%) and approaching the performance of exact backpropagation through time.
• For the pattern generation task, ModProp achieved a normalized mean squared error (NMSE) of 0.04, outperforming the baseline method (NMSE = 0.12) and demonstrating accurate temporal sequence reproduction.
• In the copy task with seven cues, ModProp achieved a test accuracy of 99.2%, indicating robust learning of long sequences with high temporal precision.
• The method showed consistent performance gains across tasks, with alignment angles between weight updates and the true gradient improving by up to 30% compared to truncated methods.
• The in-silico implementation of ModProp ran efficiently, completing one training run per task in approximately one hour on a standard server, demonstrating low computational overhead.