[Paper Review] SIT: A Bionic and Non-Linear Neuron for Spiking Neural Network

Cheng Jin, Ruijie Zhu+2 more|arXiv (Cornell University)|Mar 30, 2022

Advanced Memory and Neural Computing4 citations

TL;DR

This paper proposes the Standardized Izhikevich Tonic (SIT) neuron, a biologically plausible, non-linear spiking neuron derived from the Izhikevich model using Phase Plane Analysis (PPA) to enable integration into Spiking Neural Networks (SNNs). By hybridizing SIT with LIF neurons in a Hybrid Neural Network (HNN), the method achieves state-of-the-art performance on static and neuromorphic datasets while demonstrating spike-frequency adaptation and bursting behavior—key biological features absent in standard LIF neurons.

ABSTRACT

Spiking Neural Networks (SNNs) have piqued researchers' interest because of their capacity to process temporal information and low power consumption. However, current state-of-the-art methods limited their biological plausibility and performance because their neurons are generally built on the simple Leaky-Integrate-and-Fire (LIF) model. Due to the high level of dynamic complexity, modern neuron models have seldom been implemented in SNN practice. In this study, we adopt the Phase Plane Analysis (PPA) technique, a technique often utilized in neurodynamics field, to integrate a recent neuron model, namely, the Izhikevich neuron. Based on the findings in the advancement of neuroscience, the Izhikevich neuron model can be biologically plausible while maintaining comparable computational cost with LIF neurons. By utilizing the adopted PPA, we have accomplished putting neurons built with the modified Izhikevich model into SNN practice, dubbed as the Standardized Izhikevich Tonic (SIT) neuron. For performance, we evaluate the suggested technique for image classification tasks in self-built LIF-and-SIT-consisted SNNs, named Hybrid Neural Network (HNN) on static MNIST, Fashion-MNIST, CIFAR-10 datasets and neuromorphic N-MNIST, CIFAR10-DVS, and DVS128 Gesture datasets. The experimental results indicate that the suggested method achieves comparable accuracy while exhibiting more biologically realistic behaviors on nearly all test datasets, demonstrating the efficiency of this novel strategy in bridging the gap between neurodynamics and SNN practice.

Motivation & Objective

• To bridge the gap between neurodynamics and SNN practice by integrating biologically realistic neuron models into SNNs.
• To overcome the limitations of the Leaky-Integrate-and-Fire (LIF) neuron, which lacks spike-frequency adaptation and bursting dynamics.
• To enable the use of complex, non-linear neuron models like Izhikevich in SNNs through a systematic parameter calibration method.
• To develop a hybrid SNN architecture (HNN) combining LIF and SIT neurons to balance computational cost and biological realism.
• To validate that higher dynamic complexity in neurons improves SNN performance on both static and event-based datasets.

Proposed method

• Applies Phase Plane Analysis (PPA) to systematically analyze and calibrate the Izhikevich neuron model’s parameters (a, b, c, d) for biological plausibility and computational feasibility.
• Proposes the Standardized Izhikevich Tonic (SIT) neuron by normalizing the Izhikevich model’s dimensionless parameters using PPA, ensuring stable and reproducible dynamics.
• Designs a Hybrid Neural Network (HNN) that combines LIF and SIT neurons across separate convolutional layers to reduce computational overhead while preserving complex spiking behaviors.
• Employs a discrete-time simulation framework where membrane potential evolves via a differential equation: $ U_t = Y_{t-1} + \frac{1}{\tau}(-(U_{t-1}-U_{\text{reset}})+I_t) $, with spike reset upon threshold crossing.
• Uses spike-frequency adaptation and bursting modes as key indicators of biological plausibility, validated through simulation of input-dependent spiking patterns.
• Validates the model on both static datasets (MNIST, Fashion-MNIST, CIFAR-10) and neuromorphic datasets (N-MNIST, CIFAR10-DVS, DVS128 Gesture) using standard SNN training protocols.

Experimental results

Research questions

• RQ1 Can the Izhikevich neuron model be effectively integrated into SNNs despite its high dynamic complexity?
• RQ2 Can Phase Plane Analysis (PPA) be used to systematically determine the Izhikevich neuron’s parameters to ensure biological plausibility and computational efficiency?
• RQ3 Does the SIT neuron exhibit biologically realistic behaviors such as spike-frequency adaptation and bursting firing?
• RQ4 Can a hybrid SNN with SIT and LIF neurons outperform standard LIF-based SNNs in accuracy while maintaining low power consumption?
• RQ5 Does the increased dynamic complexity of the SIT neuron lead to improved representation learning in SNNs, as evidenced by spiking featuremaps?

Key findings

• The SIT neuron successfully exhibits spike-frequency adaptation (SFA), where firing frequency decreases after an initial peak under constant input, mimicking biological neural behavior.
• The SIT neuron can operate in bursting mode by adjusting parameters a, b, c, d, producing high signal-to-noise ratio spike bursts, a hallmark of cortical neurons.
• Compared to LIF neurons, the SIT neuron generates distinct spike patterns for different input levels (e.g., input=2 vs. input=3), while LIF neurons produce identical patterns under the same conditions.
• The SIT-based spiking featuremap shows higher firing rates and clearer edge representations than LIF-based featuremaps, indicating superior representation learning capability.
• The Hybrid Neural Network (HNN) with SIT and LIF neurons achieves comparable or better accuracy than LIF-only networks on MNIST, Fashion-MNIST, CIFAR-10, N-MNIST, CIFAR10-DVS, and DVS128 Gesture datasets.
• The study demonstrates that complex, non-linear neuron models like SIT can be effectively deployed in SNNs using PPA-based parameter calibration, overcoming prior failures in back-propagation training.