[Paper Review] Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning

Hongjian Fang, Yi Zeng+4 more|arXiv (Cornell University)|Jul 11, 2022

Advanced Memory and Neural Computing4 citations

TL;DR

This paper proposes a brain-inspired Graph Spiking Neural Network (KRR-GSNN) that uses population coding and spiking timing-dependent plasticity (STDP) to represent commonsense knowledge as a graph and perform symbolic reasoning. By integrating reward-modulated STDP (R-STDP) for reinforcement learning, the model achieves faster convergence and comparable accuracy to GCNs, with strong biological plausibility and improved interpretability through traceable reasoning paths.

ABSTRACT

How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lacks interpretability, far from how humans perceive the world. Inspired by the grandmother-cell hypothesis in neuroscience, this work investigates how population encoding and spiking timing-dependent plasticity (STDP) mechanisms can be integrated into the learning of spiking neural networks, and how a population of neurons can represent a symbol via guiding the completion of sequential firing between different neuron populations. The neuron populations of different communities together constitute the entire commonsense knowledge graph, forming a giant graph spiking neural network. Moreover, we introduced the Reward-modulated spiking timing-dependent plasticity (R-STDP) mechanism to simulate the biological reinforcement learning process and completed the related reasoning tasks accordingly, achieving comparable accuracy and faster convergence speed than the graph convolutional artificial neural networks. For the fields of neuroscience and cognitive science, the work in this paper provided the foundation of computational modeling for further exploration of the way the human brain represents commonsense knowledge. For the field of artificial intelligence, this paper indicated the exploration direction for realizing a more robust and interpretable neural network by constructing a commonsense knowledge representation and reasoning spiking neural networks with solid biological plausibility.

Motivation & Objective

• To develop a biologically plausible neural network model that represents commonsense knowledge using principles from neuroscience, such as population coding and spiking dynamics.
• To enable symbolic reasoning in spiking neural networks by encoding concepts as coordinated firing patterns across neuron populations.
• To improve interpretability and reduce the black-box nature of deep learning by making reasoning paths traceable through spiking activity.
• To explore the role of inhibitory neuron proportions in optimizing reasoning performance, inspired by the 14–15% inhibitory interneurons in the human CA1 hippocampal region.
• To model inductive reasoning in a way that mimics human-like knowledge generalization, particularly relevant to infant cognitive development.

Proposed method

• The model uses population coding, where a group of neurons collectively represents a symbol or concept, based on the grandmother-cell hypothesis.
• It employs spiking timing-dependent plasticity (STDP) to learn synaptic weights based on pre- and post-synaptic spike timing, enabling temporal coding of relationships.
• Reward-modulated STDP (R-STDP) is introduced to simulate reinforcement learning, guiding the network toward correct reasoning outcomes.
• Commonsense knowledge is encoded as a graph structure, with neuron populations representing concepts and synapses representing relationships.
• A similarity function is used to monitor network states and extract interpretable reasoning paths, enhancing model transparency.
• The model is trained via self-supervised reinforcement learning on transitivity reasoning tasks, with hyperparameter tuning on inhibitory neuron ratios.

Experimental results

Research questions

• RQ1 Can spiking neural networks with population coding and STDP mechanisms effectively represent and reason over commonsense knowledge graphs?
• RQ2 How does the proportion of inhibitory neurons in a neuron population affect the accuracy and convergence speed of reasoning tasks in a spiking network?
• RQ3 Can R-STDP in a graph-structured spiking network achieve faster convergence and comparable accuracy to traditional GCNs in symbolic reasoning tasks?
• RQ4 Does the model exhibit inductive reasoning capabilities, such as generalizing new knowledge from existing knowledge graphs?
• RQ5 To what extent does the model's architecture and dynamics reflect known biological principles of the human brain, such as the 14–15% inhibitory neuron ratio in the CA1 region?

Key findings

• The KRR-GSNN model achieved comparable final accuracy to Graph Convolutional Networks (GCNs) in transitivity reasoning tasks, but with significantly faster convergence speed.
• When the proportion of inhibitory neurons in a neuron population was set to approximately 15%, the model achieved the highest reasoning accuracy under the same number of training iterations.
• The 15% inhibitory neuron ratio closely matches the observed 14–15% in the human CA1 hippocampal region, suggesting biological plausibility of the model’s hyperparameter choice.
• The model demonstrated inductive reasoning ability by autonomously generating correct new knowledge from encoded knowledge graphs, suggesting potential for modeling human-like learning processes.
• The use of a similarity function enabled traceable reasoning paths, making the model’s internal logic interpretable and more aligned with AI ethics and safety standards.
• The integration of symbolic reasoning with biologically inspired mechanisms like STDP and R-STDP provides a computationally plausible framework for modeling human-level commonsense reasoning.