[Paper Review] Desiderata for normative models of synaptic plasticity

Colin Bredenberg, Cristina Savin|arXiv (Cornell University)|Aug 9, 2023

Neural dynamics and brain functionNeuroscience3 citations

TL;DR

This paper proposes a framework of desiderata—criteria for validity and utility—to evaluate normative models of synaptic plasticity, ensuring they link plasticity to adaptive behavior, align with biological evidence, and yield testable predictions. It analyzes REINFORCE and Wake-Sleep algorithms as case studies, showing how normative models can be grounded in optimization principles like coordinate descent or EM, though with limitations under general conditions.

ABSTRACT

Normative models of synaptic plasticity use a combination of mathematics and computational simulations to arrive at predictions of behavioral and network-level adaptive phenomena. In recent years, there has been an explosion of theoretical work on these models, but experimental confirmation is relatively limited. In this review, we organize work on normative plasticity models in terms of a set of desiderata which, when satisfied, are designed to guarantee that a model has a clear link between plasticity and adaptive behavior, consistency with known biological evidence about neural plasticity, and specific testable predictions. We then discuss how new models have begun to improve on these criteria and suggest avenues for further development. As prototypes, we provide detailed analyses of two specific models -- REINFORCE and the Wake-Sleep algorithm. We provide a conceptual guide to help develop neural learning theories that are precise, powerful, and experimentally testable.

Motivation & Objective

• To establish a rigorous, principled framework for evaluating normative models of synaptic plasticity that bridge theory and biological plausibility.
• To address the gap between theoretical models of plasticity and experimental validation by defining clear criteria for model quality.
• To demonstrate how normative models can be grounded in optimization objectives, such as maximizing likelihood or minimizing divergence.
• To critically assess canonical models like REINFORCE and Wake-Sleep in light of these criteria, identifying strengths and limitations.
• To guide future development of neural learning theories that are precise, powerful, and experimentally testable.

Proposed method

• Define a spectrum of plasticity models: phenomenological (data summary), mechanistic (biophysical causality), and normative (functional purpose).
• Propose a set of desiderata for normative models: clear link to adaptive behavior, consistency with biological evidence, and testable predictions.
• Use REINFORCE as a case study to show how policy gradient methods can be framed as normative plasticity rules optimizing behavioral objectives.
• Analyze the Wake-Sleep algorithm as a normative model that approximates coordinate descent or EM under specific assumptions.
• Derive mathematical conditions under which Wake-Sleep updates approximate gradient descent on a joint likelihood objective.
• Apply first-order Taylor approximation to show equivalence between Wake and Sleep phase gradients when the model is near convergence.

Figure 1: Defining normative modeling. a. Spectrum of synaptic plasticity models. Mechanistic models show how detailed biophysical interactions produce observed plasticity, phenomenological models concisely describe what changes in experimental variables (e.g. post-pre relative spike timing $\Delta

Experimental results

Research questions

• RQ1 What criteria define a high-quality normative model of synaptic plasticity that is both biologically plausible and behaviorally meaningful?
• RQ2 How can normative models be formally linked to optimization objectives such as likelihood maximization or reward maximization?
• RQ3 In what conditions does the Wake-Sleep algorithm approximate coordinate descent or the EM algorithm?
• RQ4 Why do some normative models, like Wake-Sleep, fail to converge reliably despite empirical success?
• RQ5 How can we ensure that normative plasticity models generate specific, falsifiable predictions for experimental testing?

Key findings

• The Wake-Sleep algorithm approximates coordinate descent on a joint likelihood objective when the model is near convergence, as shown by gradient equivalence between Wake and Sleep phases.
• Under the assumption that the model distribution $ p_{m} $ closely approximates the true data distribution $ p $, the gradients of the Wake and Sleep objectives become approximately equal.
• The Wake-Sleep algorithm can be interpreted as an approximation of the EM algorithm when the generative model is convex and the global minimum is attainable.
• REINFORCE provides a normative framework for reinforcement learning by using policy gradients to optimize behavior, with updates derived from the expected reward gradient.
• Despite empirical success, the Wake-Sleep algorithm lacks strong convergence guarantees under general conditions, especially when the model is non-convex or the global minimum is not reachable.
• The desiderata framework enables systematic evaluation of plasticity models by ensuring they connect plasticity to adaptive function, respect biological constraints, and produce testable predictions.

Figure 2: Architecture and scalability considerations for normative plasticity models. a. Features of realistic biological networks that normative plasticity theories should be able to account for: separation of excitatory and inhibitory neuron populations; stochastic and spiking input-output functi