[Paper Review] A Novel Neuromorphic Processors Realization of Spiking Deep Reinforcement Learning for Portfolio Management

Seyyed Amirhossein Saeidi, Forouzan Fallah+2 more|arXiv (Cornell University)|Mar 26, 2022

Advanced Memory and Neural Computing4 citations

TL;DR

This paper proposes a spiking deep reinforcement learning (SDRL) algorithm optimized for Intel's Loihi neuromorphic processor to enhance portfolio management in financial markets. By leveraging event-based computation, the method achieves 186x–516x lower energy consumption and 1.3x–2.0x speedup over conventional high-end CPUs and GPUs, respectively, while effectively managing risk and maximizing returns in volatile cryptocurrency markets between 2016 and 2021.

ABSTRACT

The process of continuously reallocating funds into financial assets, aiming to increase the expected return of investment and minimizing the risk, is known as portfolio management. Processing speed and energy consumption of portfolio management have become crucial as the complexity of their real-world applications increasingly involves high-dimensional observation and action spaces and environment uncertainty, which their limited onboard resources cannot offset. Emerging neuromorphic chips inspired by the human brain increase processing speed by up to 1000 times and reduce power consumption by several orders of magnitude. This paper proposes a spiking deep reinforcement learning (SDRL) algorithm that can predict financial markets based on unpredictable environments and achieve the defined portfolio management goal of profitability and risk reduction. This algorithm is optimized forIntel's Loihi neuromorphic processor and provides 186x and 516x energy consumption reduction is observed compared to the competitors, respectively. In addition, a 1.3x and 2.0x speed-up over the high-end processors and GPUs, respectively. The evaluations are performed on cryptocurrency market between 2016 and 2021 the benchmark.

Motivation & Objective

• To address the growing computational and energy demands of high-dimensional, uncertain financial environments in portfolio management.
• To reduce energy consumption and increase processing speed in real-time financial decision-making systems.
• To develop a spiking deep reinforcement learning algorithm natively optimized for neuromorphic hardware like Intel Loihi.
• To evaluate the SDRL approach on real-world cryptocurrency data from 2016 to 2021 for profitability and risk reduction.

Proposed method

• The authors design a spiking deep reinforcement learning (SDRL) algorithm that processes financial time series data using event-driven spiking neurons.
• The algorithm is compiled and executed on Intel's Loihi neuromorphic processor, leveraging its asynchronous, low-power computation model.
• Neural network layers are converted into spiking neural networks (SNNs) with temporal coding to represent market features and actions.
• Reinforcement learning policy gradients are adapted to work with spike-based backpropagation for training the SNN policy.
• The system uses a reward shaping mechanism to balance return maximization and risk minimization in portfolio allocation.
• The architecture is evaluated using a benchmark of cryptocurrency price data (2016–2021) under real-time trading constraints.

Experimental results

Research questions

• RQ1 Can a spiking deep reinforcement learning model effectively learn optimal portfolio allocation strategies in high-dimensional, uncertain financial environments?
• RQ2 How does the energy efficiency and inference speed of a neuromorphic-optimized SDRL system compare to conventional CPU and GPU implementations?
• RQ3 To what extent can event-based spiking computation maintain performance in dynamic financial time series prediction and trading?
• RQ4 Does the SDRL model achieve superior risk-adjusted returns compared to baseline reinforcement learning models in volatile cryptocurrency markets?
• RQ5 How well does the Loihi neuromorphic processor scale for real-time, low-latency portfolio management applications?

Key findings

• The proposed SDRL algorithm achieves 186x and 516x reductions in energy consumption compared to CPU and GPU baselines, respectively.
• The system delivers a 1.3x speed-up over high-end conventional CPUs and a 2.0x speed-up over GPUs in inference performance.
• The algorithm successfully learns to balance profitability and risk, achieving stable portfolio returns across volatile cryptocurrency markets from 2016 to 2021.
• The neuromorphic implementation maintains high accuracy in market prediction and action selection despite sparse, asynchronous spike-based processing.
• The Loihi-optimized SDRL framework demonstrates strong scalability and low-latency performance suitable for real-time financial decision systems.
• The results confirm that neuromorphic hardware significantly enhances the efficiency of deep reinforcement learning in financial applications.