[Paper Review] Biologically Inspired Dynamic Thresholds for Spiking Neural Networks

Jianchuan Ding, Bo Dong+5 more|arXiv (Cornell University)|Jun 9, 2022

Advanced Memory and Neural ComputingEngineering22 citations

TL;DR

Introduces a bioinspired dynamic energy-temporal threshold (BDETT) for SNNs that couples dynamic energy and temporal thresholds to regulate firing, improving generalization and homeostasis in robotic obstacle avoidance, continuous control, and image classification under normal and degraded conditions.

ABSTRACT

The dynamic membrane potential threshold, as one of the essential properties of a biological neuron, is a spontaneous regulation mechanism that maintains neuronal homeostasis, i.e., the constant overall spiking firing rate of a neuron. As such, the neuron firing rate is regulated by a dynamic spiking threshold, which has been extensively studied in biology. Existing work in the machine learning community does not employ bioinspired spiking threshold schemes. This work aims at bridging this gap by introducing a novel bioinspired dynamic energy-temporal threshold (BDETT) scheme for spiking neural networks (SNNs). The proposed BDETT scheme mirrors two bioplausible observations: a dynamic threshold has 1) a positive correlation with the average membrane potential and 2) a negative correlation with the preceding rate of depolarization. We validate the effectiveness of the proposed BDETT on robot obstacle avoidance and continuous control tasks under both normal conditions and various degraded conditions, including noisy observations, weights, and dynamic environments. We find that the BDETT outperforms existing static and heuristic threshold approaches by significant margins in all tested conditions, and we confirm that the proposed bioinspired dynamic threshold scheme offers homeostasis to SNNs in complex real-world tasks.

Motivation & Objective

• Motivate the need for dynamic, bioinspired regulation of spiking thresholds to achieve neuronal homeostasis in SNNs.
• Propose the BDETT scheme comprising dynamic energy threshold (DET) and dynamic temporal threshold (DTT) to regulate firing.
• Show how layerwise statistical cues set DET and DTT parameters for robust generalization.
• Validate BDETT on obstacle avoidance, continuous control, and image classification under normal and degraded conditions.

Proposed method

• Define a neuronwise dynamic threshold Theta_i^l(t+1) = 1/2 (E_i^l(t) + T_i^l(t+1)).
• Dynamic Energy Threshold (DET) E_i^l(t) uses a bioinspired form combining v_i^l(t), layer means V_m^l(t) and V_theta^l(t) with layerwise statistics (Equations 2-4).
• Dynamic Temporal Threshold (DTT) T_i^l(t+1) uses a monoexponential-like function linked to the prior rate of depolarization with an adaptive offset a depending on layer thresholds (Equations 5-6).
• E_i^l(t) and V_m^l(t), V_theta^l(t) incorporate layerwise mean and range to adapt DET to the layer statistics.
• Integrate BDETT into SRM and LIF SNNs and evaluate on obstacle avoidance, continuous control (HalfCheetah-v3, Ant-v3), and image classification under degraded inputs and weight uncertainty.

Experimental results

Research questions

• RQ1 Can a biologically inspired dynamic threshold scheme improve the generalization of SNNs in real-world robotic tasks?
• RQ2 Do dynamic energy and temporal thresholds (DET and DTT) provide homeostasis that enhances robustness under degraded inputs, noisier weights, and dynamic environments?
• RQ3 How does BDETT compare to static and heuristic dynamic thresholds across obstacle avoidance, robotics control, and perception tasks?

Key findings

• BDETT outperforms static and heuristic dynamic thresholds across all tested tasks and degraded conditions.
• BDETT provides measurable homeostasis to host SNNs, with smaller changes in firing-rate statistics when conditions shift.
• In obstacle avoidance, BDETT yields higher success rates under dynamic obstacles, degraded inputs, and weight uncertainty.
• In continuous control tasks, BDETT achieves higher rewards under degraded observations and weight perturbations, maintaining better performance than baselines.
• BDETT remains effective with low-precision weights, sometimes outperforming high-precision baselines on certain tasks.