[Paper Review] Differentiable plasticity: training plastic neural networks with backpropagation

Thomas Miconi, Jeff Clune+1 more|arXiv (Cornell University)|Apr 6, 2018

Advanced Memory and Neural Computing92 citations

TL;DR

The paper trains neural networks with trainable plasticity (Hebbian traces) via gradient descent, enabling fast lifelong learning across pattern memorization, one-shot Omniglot classification, and maze reinforcement tasks, often outperforming non-plastic baselines.

ABSTRACT

How can we build agents that keep learning from experience, quickly and efficiently, after their initial training? Here we take inspiration from the main mechanism of learning in biological brains: synaptic plasticity, carefully tuned by evolution to produce efficient lifelong learning. We show that plasticity, just like connection weights, can be optimized by gradient descent in large (millions of parameters) recurrent networks with Hebbian plastic connections. First, recurrent plastic networks with more than two million parameters can be trained to memorize and reconstruct sets of novel, high-dimensional 1000+ pixels natural images not seen during training. Crucially, traditional non-plastic recurrent networks fail to solve this task. Furthermore, trained plastic networks can also solve generic meta-learning tasks such as the Omniglot task, with competitive results and little parameter overhead. Finally, in reinforcement learning settings, plastic networks outperform a non-plastic equivalent in a maze exploration task. We conclude that differentiable plasticity may provide a powerful novel approach to the learning-to-learn problem.

Motivation & Objective

• Motivate lifelong learning through synaptic plasticity inspired by biology.
• Introduce a differentiable plasticity framework where each connection has a fixed and a plastic component.
• Demonstrate gradient-based optimization of plasticity parameters on diverse tasks (pattern memorization, Omniglot, reinforcement learning).
• Show that learned plasticity can outperform non-plastic counterparts on complex memory tasks and compete in meta-learning benchmarks.

Proposed method

• Define a network where each connection has a fixed weight w_ij and a plastic component alpha_ij * Hebb_ij, with Hebb_ij tracing pre- and post-synaptic activity.
• Use a recurrent update for Hebb_ij, e.g. Hebb_ij(t+1) = eta * x_i(t-1) * x_j(t) + (1 - eta) * Hebb_ij(t) (or alternative Oja’s rule variant without decay).
• Total effective weight is w_ij + alpha_ij * Hebb_ij, enabling both fixed and plastic paths.
• Optimize w_ij and alpha_ij by backpropagation through episodes; eta (plasticity learning rate) is shared and learned.
• Test across tasks: binary pattern memorization, natural image memorization (CIFAR-10), Omniglot one-shot classification, and a maze exploration RL task.

Experimental results

Research questions

• RQ1 Can differentiable plasticity be trained with backpropagation at scale (millions of parameters)?
• RQ2 Does learned plasticity enable rapid memory formation and pattern reconstruction without relying on fixed weights alone?
• RQ3 Is differentiable plasticity competitive with established meta-learning approaches on Omniglot and reinforcement learning tasks?
• RQ4 How does per-connection plasticity structure (independent alpha_ij) compare to shared plasticity in performance?
• RQ5 Can plastic networks outperform non-plastic recurrent models (RNNs/LSTMs) on complex memorization tasks?

Key findings

• Plastic networks with differentiable plasticity solve high-dimensional pattern memorization that non-plastic RNNs and LSTMs struggle with, often faster (e.g., 2,000 vs 500,000 episodes).
• On natural image memorization, learned plasticity yields structured Hebb matrices and outperforms fixed-plasticity baselines, with independent alphas enhancing performance over shared alpha.
• Omniglot 5-way 1-shot classification achieves 98.3% accuracy (95% CI ±0.80) using plastic connections, competitive with several meta-learning methods and requiring only modest parameter overhead.
• In a maze reinforcement learning task, differentiable plasticity improves performance over non-plastic and homogeneously plastic networks, indicating benefits of per-connection plasticity customization.
• The study demonstrates that gradient descent can optimize plasticity rules themselves, enabling effective meta-learning and memory-enhanced computation beyond traditional fixed-weight networks.