[Paper Review] Dendritic cortical microcircuits approximate the backpropagation algorithm

João Sacramento, Rui Ponte Costa+2 more|arXiv (Cornell University)|Oct 26, 2018

Neural dynamics and brain function93 citations

TL;DR

The paper proposes a multi-compartment cortical microcircuit model where error signals are encoded in apical dendrites and learning happens continuously, approximating backpropagation without distinct phases.

ABSTRACT

Deep learning has seen remarkable developments over the last years, many of them inspired by neuroscience. However, the main learning mechanism behind these advances - error backpropagation - appears to be at odds with neurobiology. Here, we introduce a multilayer neuronal network model with simplified dendritic compartments in which error-driven synaptic plasticity adapts the network towards a global desired output. In contrast to previous work our model does not require separate phases and synaptic learning is driven by local dendritic prediction errors continuously in time. Such errors originate at apical dendrites and occur due to a mismatch between predictive input from lateral interneurons and activity from actual top-down feedback. Through the use of simple dendritic compartments and different cell-types our model can represent both error and normal activity within a pyramidal neuron. We demonstrate the learning capabilities of the model in regression and classification tasks, and show analytically that it approximates the error backpropagation algorithm. Moreover, our framework is consistent with recent observations of learning between brain areas and the architecture of cortical microcircuits. Overall, we introduce a novel view of learning on dendritic cortical circuits and on how the brain may solve the long-standing synaptic credit assignment problem.

Motivation & Objective

• Motivate a biologically plausible solution to the synaptic credit assignment problem in deep networks.
• Propose a multilayer network with dendritic compartments that encode errors for learning.
• Demonstrate that continuous, local plasticity approximates backpropagation analytically and empirically.
• Show the model can perform nonlinear regression and MNIST digit classification.
• Relate the framework to cortical microcircuit architecture and predictive coding theories.

Proposed method

• Use a 3-compartment model for pyramidal neurons with somatic, basal, and apical dendritic compartments.
• Introduce interneurons that cancel top-down input to cleanly encode apical errors.
• Define synaptic learning rules that depend on local dendritic prediction errors and presynaptic activity (Eqs. 6–10).
• Demonstrate self-predicting network states where apical activity is canceled in the absence of teaching signals.
• Provide analytic derivations showing how updates approximate backprop in the weak-feedback limit and via feedback alignment.
• Empirically evaluate on nonlinear regression and MNIST classification, with continuous learning and no phase-based learning.

Experimental results

Research questions

• RQ1 Can dendritic compartments encode prediction errors that drive learning of bottom-up connections in a biologically plausible network?
• RQ2 Do continuous, local synaptic plasticity rules in a multi-layer dendritic circuit approximate backpropagation?
• RQ3 How do lateral SST interneurons and top-down feedback contribute to error signaling and learning without requiring separate phases?
• RQ4 Can such a network learn nonlinear mappings and handle real-world tasks like MNIST classification?
• RQ5 What is the role of feedback alignment and top-down weight symmetry in achieving backprop-like learning?],
• RQ6 key_findings_list_start_1s_empty_
• RQ7 key_findingsAdditionally_
• RQ8 key_findings możliwości
• RQ9 key_findings_again
• RQ10 key_findings_empty_headers

Key findings

• The model’s apical dendrites encode errors from mismatch between lateral interneuron input and top-down feedback, driving local plasticity to minimize error.
• Analytically, under self-predicting conditions and small feedback strength, bottom-up weight updates converge to backpropagation updates (modulo a scaling factor and weight symmetry or alignment constraints).
• The network learns nonlinear regression tasks with continuous learning and without alternating learning phases, achieving competitive performance.
• In MNIST classification with a four-layer network, the model attains a test error of 1.96%, approaching backprop-based non-convolutional networks.
• The approach supports predictions about error propagation across cortical areas and aligns with observed SST interneuron roles and predictive-coding interpretations.
• The model remains robust to symmetry-breaking between forward and feedback weights via feedback alignment.