[Paper Review] Brain-inspired global-local hybrid learning towards human-like intelligence.

Yujie Wu, Rong Zhao+12 more|arXiv (Cornell University)|Jun 5, 2020

Advanced Memory and Neural Computing46 references4 citations

TL;DR

This paper proposes a brain-inspired hybrid learning model that unifies neuroscience and machine learning via a spiking neural network with a meta-local plasticity module and two-phase parametric learning. The model enables energy-efficient, multi-scale learning across tasks like few-shot and fault-tolerant learning, validated on the Tianjic neuromorphic platform.

ABSTRACT

Two main routes of learning methods exist at present including neuroscience-inspired methods and machine learning methods. Both have own advantages, but neither currently can solve all learning problems well. Integrating them into one network may provide better learning abilities for general tasks. On the other hand, spiking neural network embodies computation in spatiotemporal domain with unique features of rich coding scheme and threshold switching, which is very suitable for low power and high parallel neuromorphic computing. Here, we report a spike-based general learning model that integrates two learning routes by introducing a brain-inspired meta-local module and a two-phase parametric modelling. The hybrid model can meta-learn general local plasticity, and receive top-down supervision information for multi-scale learning. We demonstrate that this hybrid model facilitates learning of many general tasks, including fault-tolerance learning, few-shot learning and multiple-task learning. Furthermore, the implementation of the hybrid model on the Tianjic neuromorphic platform proves that it can fully utilize the advantages of neuromorphic hardware architecture and promote energy-efficient on-chip applications.

Motivation & Objective

• To address the limitations of standalone neuroscience-inspired and machine learning methods in solving general learning tasks.
• To integrate brain-inspired meta-local plasticity with parametric modeling for adaptive, multi-scale learning.
• To enable top-down supervision in spiking neural networks for improved generalization across diverse tasks.
• To leverage neuromorphic hardware advantages for low-power, on-chip inference and learning.
• To demonstrate energy-efficient implementation of general-purpose learning on the Tianjic platform.

Proposed method

• Introduces a brain-inspired meta-local module that enables meta-learning of general local plasticity rules in spiking neural networks.
• Employs a two-phase parametric modeling approach to support both online adaptation and top-down supervision.
• Utilizes spatiotemporal coding and threshold switching in spiking neurons to enable low-power, high-parallel computation.
• Designs the hybrid model to receive top-down feedback for multi-scale learning and improved task generalization.
• Maps the hybrid model onto the Tianjic neuromorphic platform to exploit its hardware-native support for spiking dynamics.
• Leverages the neuromorphic architecture to achieve energy-efficient on-chip learning and inference.

Experimental results

Research questions

• RQ1 Can a hybrid learning model that integrates neuroscience-inspired and machine learning principles achieve better generalization across diverse tasks?
• RQ2 How can meta-learned local plasticity rules enhance adaptability in spiking neural networks?
• RQ3 To what extent can top-down supervision improve multi-scale learning in spike-based models?
• RQ4 Can such a hybrid model achieve energy-efficient on-chip learning on neuromorphic hardware?
• RQ5 How does the integration of spiking dynamics and parametric modeling support fault-tolerant and few-shot learning?

Key findings

• The hybrid model successfully facilitates fault-tolerant learning by adapting to noisy or incomplete input data through meta-learned plasticity.
• The model achieves effective few-shot learning by leveraging meta-learned local plasticity and top-down feedback for rapid adaptation.
• Multi-task learning is enhanced through the two-phase parametric modeling, enabling efficient transfer across tasks.
• The implementation on the Tianjic neuromorphic platform confirms the model’s compatibility with hardware-native spiking dynamics.
• The model demonstrates energy efficiency by fully utilizing the neuromorphic hardware architecture for on-chip learning applications.
• The integration of brain-inspired mechanisms with parametric learning enables robust performance across general learning tasks.