[Paper Review] Efficient and Effective Time-Series Forecasting with Spiking Neural Networks

Changze Lv, Yansen Wang+4 more|arXiv (Cornell University)|Feb 2, 2024

Time Series Analysis and Forecasting4 citations

TL;DR

This paper proposes a unified framework for efficient and accurate time-series forecasting using spiking neural networks (SNNs), leveraging event-driven computation and spike encoding to achieve performance on par with or better than traditional artificial neural networks (ANNs), while reducing energy consumption by up to 75.05%. The approach enables biologically plausible, low-power temporal modeling of sequential data.

ABSTRACT

Spiking neural networks (SNNs), inspired by the spiking behavior of biological neurons, provide a unique pathway for capturing the intricacies of temporal data. However, applying SNNs to time-series forecasting is challenging due to difficulties in effective temporal alignment, complexities in encoding processes, and the absence of standardized guidelines for model selection. In this paper, we propose a framework for SNNs in time-series forecasting tasks, leveraging the efficiency of spiking neurons in processing temporal information. Through a series of experiments, we demonstrate that our proposed SNN-based approaches achieve comparable or superior results to traditional time-series forecasting methods on diverse benchmarks with much less energy consumption. Furthermore, we conduct detailed analysis experiments to assess the SNN's capacity to capture temporal dependencies within time-series data, offering valuable insights into its nuanced strengths and effectiveness in modeling the intricate dynamics of temporal data. Our study contributes to the expanding field of SNNs and offers a promising alternative for time-series forecasting tasks, presenting a pathway for the development of more biologically inspired and temporally aware forecasting models. Our code is available at https://github.com/microsoft/SeqSNN.

Motivation & Objective

• To address the underexplored potential of spiking neural networks (SNNs) in time-series forecasting, where SNNs' event-driven nature is poorly aligned with continuous sequential data.
• To overcome challenges in temporal alignment and spike encoding that hinder SNN application to time-series data.
• To develop a standardized, effective framework for SNN-based time-series forecasting, including data encoding and model architecture selection.
• To demonstrate that SNNs can match or exceed the predictive accuracy of ANNs while drastically reducing energy consumption in inference.

Proposed method

• Proposes a unified framework integrating time-series data encoding, SNN model adaptation, and energy-efficient inference for forecasting tasks.
• Designs two distinct encoding layers to convert continuous time-series values into biologically plausible spike trains, preserving temporal dynamics.
• Adapts three core deep learning architectures—TCN, RNN, and Transformer—into their SNN equivalents (Spike-TCN, Spike-RNN, iSpikformer) with no floating-point operations.
• Employs a neuromorphic hardware model (45nm) to estimate theoretical energy consumption, comparing SNNs against their ANN counterparts.
• Introduces a beta parameter in the SNN neuron model to control membrane potential decay, enabling control over temporal memory persistence.
• Conducts extensive ablation and analysis on synthetic and real-world benchmarks to evaluate temporal dependency modeling and robustness.

Experimental results

Research questions

• RQ1 Can SNNs achieve comparable or superior forecasting performance to ANNs on standard time-series benchmarks while consuming significantly less energy?
• RQ2 How do different SNN architectures (Spike-TCN, Spike-RNN, iSpikformer) perform in capturing long- and short-term temporal dependencies in time-series data?
• RQ3 What is the impact of spike encoding strategies and model hyperparameters (e.g., beta) on forecasting accuracy and energy efficiency?
• RQ4 To what extent can SNNs model complex temporal dynamics in both low- and high-frequency synthetic signals?

Key findings

• SNN-based models achieve comparable or better R² performance than their ANN counterparts on four diverse time-series benchmarks, including Electricity, Metr-La, Solar, and ETTh.
• The proposed SNN models reduce energy consumption by an average of 70.33% compared to ANNs, with Spike-GRU showing up to 75.05% reduction in energy use.
• Spike-TCN shows slightly lower accuracy in predicting peak values of high-frequency signals compared to Spike-RNN and iSpikformer, indicating architectural sensitivity to signal frequency.
• The SNN framework successfully captures temporal dependencies in both low- and high-frequency synthetic signals, demonstrating robustness across signal dynamics.
• Increasing the beta parameter enhances long-term memory retention, improving performance on datasets with long-range dependencies such as Metr-La and Solar.
• The energy efficiency of SNNs is primarily driven by sparse spike activity and reduced computational operations, particularly in models like Spike-GRU with low average firing rates.