[Paper Review] A Spiking Neural Network for Image Segmentation

Kinjal A. Patel, Eric Hunsberger+2 more|arXiv (Cornell University)|Jun 16, 2021

Advanced Memory and Neural Computing14 references37 citations

TL;DR

The paper converts U-Net to a spiking neural network implemented on Intel Loihi, trains rate- and spike-based models, and demonstrates over 2x energy efficiency with comparable accuracy on a Loihi-based pipeline using 8-bit quantization.

ABSTRACT

We seek to investigate the scalability of neuromorphic computing for computer vision, with the objective of replicating non-neuromorphic performance on computer vision tasks while reducing power consumption. We convert the deep Artificial Neural Network (ANN) architecture U-Net to a Spiking Neural Network (SNN) architecture using the Nengo framework. Both rate-based and spike-based models are trained and optimized for benchmarking performance and power, using a modified version of the ISBI 2D EM Segmentation dataset consisting of microscope images of cells. We propose a partitioning method to optimize inter-chip communication to improve speed and energy efficiency when deploying multi-chip networks on the Loihi neuromorphic chip. We explore the advantages of regularizing firing rates of Loihi neurons for converting ANN to SNN with minimum accuracy loss and optimized energy consumption. We propose a percentile based regularization loss function to limit the spiking rate of the neuron between a desired range. The SNN is converted directly from the corresponding ANN, and demonstrates similar semantic segmentation as the ANN using the same number of neurons and weights. However, the neuromorphic implementation on the Intel Loihi neuromorphic chip is over 2x more energy-efficient than conventional hardware (CPU, GPU) when running online (one image at a time). These power improvements are achieved without sacrificing the task performance accuracy of the network, and when all weights (Loihi, CPU, and GPU networks) are quantized to 8 bits.

Motivation & Objective

• Investigate scalable neuromorphic approaches for computer vision with competitive performance and reduced power.
• Adapt a deep segmentation architecture (U-Net) to a spiking framework using the Nengo toolchain.
• Evaluate rate-based and spike-based SNN implementations and quantify energy efficiency vs. conventional hardware.
• Develop methods to optimize inter-chip communication and neuron firing rates for Loihi deployments.

Proposed method

• Convert a downsized U-Net (238K neurons) from ANN to SNN using Nengo/NengoLoihi with a rate-based ANN-to-SNN conversion and a spike-based implementation for inference.
• Train the network with an ANN framework (TensorFlow) and transfer to NengoDL/NengoLoihi, applying a specialized forward nonlinearity to match Loihi’s integrate-and-fire neurons.
• Introduce a percentile-based firing-rate regularization loss to constrain neuron firing within 50–200 Hz to balance accuracy and energy.
• Propose a partitioning algorithm to place the model across Loihi cores and minimize inter-chip communication for multi-chip deployments.
• Quantize weights to 8-bit and design a weight scaling and exponent scheme to map floating-point weights to Loihi’s fixed-point representation.
• Benchmark energy and accuracy on Loihi versus CPU/GPU for online (one image at a time) and parallel processing.

Experimental results

Research questions

• RQ1 Can a U-Net like segmentation network be effectively converted to an SNN without significant loss in segmentation accuracy?
• RQ2 What regularization and architectural strategies best balance accuracy and energy efficiency on Loihi for image segmentation?
• RQ3 How does multi-chip partitioning impact latency and energy consumption for large neuromorphic segmentation networks?
• RQ4 What are practical quantization and mapping methods from floating-point ANN weights to Loihi’s fixed-point model that preserve performance?
• RQ5 How do rate-based and spike-based implementations compare in accuracy and power for neuromorphic image segmentation?

Key findings

• The SNN achieves similar semantic segmentation performance to the ANN when using the same number of neurons and weights.
• Loihi-based inference is over 2x more energy-efficient than CPU/GPU for online processing with 8-bit quantized weights.
• A percentile-based firing-rate regularization keeps neuron firing within 50–200 Hz, balancing accuracy and energy.
• A two-chip Loihi deployment with a scaled-down U-Net (238K neurons) demonstrates scalability for neuromorphic image segmentation.
• A partitioning method minimizes inter-chip communication, improving speed and energy efficiency in multi-chip Loihi deployments.