[Paper Review] Online Training Through Time for Spiking Neural Networks

Mingqing Xiao, Qingyan Meng+3 more|arXiv (Cornell University)|Oct 9, 2022

Advanced Memory and Neural Computing29 citations

TL;DR

The paper introduces Online Training Through Time (OTTT) for SNNs, enabling forward-in-time learning with constant memory and connecting BPTT with surrogate gradients to spike representations, backed by theoretical descent guarantees and Hebbian-like updates.

ABSTRACT

Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Recent progress in training methods has enabled successful deep SNNs on large-scale tasks with low latency. Particularly, backpropagation through time (BPTT) with surrogate gradients (SG) is popularly used to achieve high performance in a very small number of time steps. However, it is at the cost of large memory consumption for training, lack of theoretical clarity for optimization, and inconsistency with the online property of biological learning and rules on neuromorphic hardware. Other works connect spike representations of SNNs with equivalent artificial neural network formulation and train SNNs by gradients from equivalent mappings to ensure descent directions. But they fail to achieve low latency and are also not online. In this work, we propose online training through time (OTTT) for SNNs, which is derived from BPTT to enable forward-in-time learning by tracking presynaptic activities and leveraging instantaneous loss and gradients. Meanwhile, we theoretically analyze and prove that gradients of OTTT can provide a similar descent direction for optimization as gradients based on spike representations under both feedforward and recurrent conditions. OTTT only requires constant training memory costs agnostic to time steps, avoiding the significant memory costs of BPTT for GPU training. Furthermore, the update rule of OTTT is in the form of three-factor Hebbian learning, which could pave a path for online on-chip learning. With OTTT, it is the first time that two mainstream supervised SNN training methods, BPTT with SG and spike representation-based training, are connected, and meanwhile in a biologically plausible form. Experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS demonstrate the superior performance of our method on large-scale static and neuromorphic datasets in small time steps.

Motivation & Objective

• Motivate efficient training of spiking neural networks (SNNs) with low latency and online learning properties.
• Develop a training method that avoids the high memory cost of BPTT while preserving performance.
• Establish theoretical connections between OTTT and spike-representation based methods to ensure descent directions.
• Show that OTTT aligns with biologically plausible three-factor Hebbian learning for on-chip learning.
• Demonstrate effectiveness on large-scale static and neuromorphic datasets.

Proposed method

• Derive OTTT from BPTT by decoupling temporal dependencies and tracking presynaptic activities to enable forward-in-time gradient computation.
• Compute instantaneous loss and gradients at each time step, avoiding backpropagation through time.
• Prove that OTTT gradients provide similar descent directions to spike-representation based gradients under feedforward and recurrent conditions.
• Formulate the update as a three-factor Hebbian learning rule, enabling potential on-chip online learning.
• Implement with noBatchNorm using scaled weight standardization (sWS) to support normalization-free training on neuromorphic hardware.
• Provide both online (OTTT O) and accumulated-gradient (OTTT A) variants and detailed implementation guidance.

Experimental results

Research questions

• RQ1 Can online, forward-in-time learning for SNNs achieve competitive performance with constant memory costs compared to BPTT?
• RQ2 Do OTTT gradients provide a descent direction similar to gradients derived from spike representations under both feedforward and recurrent dynamics?
• RQ3 Is OTTT compatible with a biologically plausible three-factor Hebbian update scheme suitable for on-chip learning?
• RQ4 How does OTTT perform on large-scale static and neuromorphic datasets with low time-step latency?

Key findings

• OTTT achieves competitive accuracy with a small number of time steps on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS.
• OTTT reduces training memory from linear growth with time steps (BPTT) to a constant memory cost, enabling larger effective batch sizes at the same resources.
• Theoretical analysis shows OTTT gradients provide a descent direction comparable to spike-representation based gradients under convergent inputs for both feedforward and recurrent networks.
• OTTT updates can be viewed as a three-factor Hebbian learning rule, suggesting feasibility for online on-chip learning on neuromorphic hardware.
• Empirical results demonstrate strong performance advantages over several baselines (including some BPTT variants) with as few as 6 time steps in many cases.
• The method bridges BPTT with SG and spike-representation based training, while maintaining online learning suitable for neuromorphic systems.