[Paper Review] EEG-Based Emotion Recognition Using Regularized Graph Neural Networks

Peixiang Zhong, Di Wang+1 more|arXiv (Cornell University)|Jul 18, 2019

EEG and Brain-Computer Interfaces79 references47 citations

TL;DR

This paper proposes a regularized graph neural network (RGNN) for EEG-based emotion recognition that leverages biological brain topology and two regularizers to handle cross-subject variation and noisy labels, achieving superior performance on SEED and SEED-IV datasets.

ABSTRACT

Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes. Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels. In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition. RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels. Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization. In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively. Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings. Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model. Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition.

Motivation & Objective

• Motivate emotion recognition from EEG signals and address limitations of traditional topology-insensitive methods.
• Introduce a biologically inspired, sparse adjacency matrix to model inter-channel brain topology in a graph neural network.
• Develop two regularizers, NodeDAT and EmotionDL, to improve cross-subject generalization and robustness to noisy labels.
• Demonstrate RGNN's effectiveness through extensive experiments on SEED and SEED-IV datasets and analyze informative brain regions.
• Provide insights into inter-channel relations and brain areas relevant to EEG-based emotion recognition.

Proposed method

• Model EEG channels as nodes in a graph with a learnable, symmetric adjacency matrix capturing local and global brain connections.
• Extend Simple Graph Convolution (SGC) to RGNN to exploit topological structure and obtain node-aware representations.
• Introduce NodeDAT to perform node-wise domain adversarial training and regularize feature representations per channel for cross-subject robustness.
• Introduce EmotionDL to learn a distribution over emotion labels for each sample, using KL divergence to handle noisy labels.
• Optimize end-to-end with a loss combining KL-based distribution learning, sparsity on the adjacency matrix, and domain-adversarial regularization.
• Initialize adjacency with distance-based local connections and specific global channel pairs to reflect neural asymmetry and functional connectivity.

Experimental results

Research questions

• RQ1 Can a graph neural network that respects EEG channel topology improve emotion recognition accuracy over topology-agnostic models?
• RQ2 Does a biologically inspired adjacency matrix improve learning of inter-channel relations in EEG-based emotion recognition?
• RQ3 Do NodeDAT and EmotionDL regularizers improve cross-subject generalization and robustness to noisy labels, respectively?
• RQ4 What neural regions and inter-channel relationships are most informative for EEG-based emotion classification?
• RQ5 How does RGNN perform on standard EEG emotion datasets SEED and SEED-IV in both subject-dependent and subject-independent settings?

Key findings

• RGNN with the proposed adjacency and regularizers achieves superior performance compared to state-of-the-art models in most experimental settings.
• Ablation studies show that the adjacency matrix design and the two regularizers consistently and significantly contribute to performance gains.
• NodeDAT improves cross-subject generalization by providing finer-grained domain adaptation at the node level.
• EmotionDL enhances robustness to noisy labels by learning a distribution over emotion classes rather than a single hard label.
• Analyses indicate that pre-frontal, parietal, and occipital regions, as well as global left-right hemisphere connections, are informative for emotion recognition.