[论文解读] Self-supervised Group Meiosis Contrastive Learning for EEG-Based Emotion Recognition

Haoning Kan, Jiale Yu+3 more|arXiv (Cornell University)|Jul 12, 2022

EEG and Brain-Computer Interfaces被引用 6

一句话总结

该论文提出了一种自监督的群体减数分裂对比学习（SGMC）框架，用于基于EEG的情绪识别，该框架利用了群体EEG样本间的刺激一致性。通过引入一种新颖的减数分裂启发式数据增强方法以及用于学习共享表征的群体投影器，SGMC在DEAP数据集上实现了94.72%和95.68%的最先进准确率，分别对应效价和唤醒度，即使在标注数据有限的情况下也表现优异。

ABSTRACT

The progress of EEG-based emotion recognition has received widespread attention from the fields of human-machine interactions and cognitive science in recent years. However, how to recognize emotions with limited labels has become a new research and application bottleneck. To address the issue, this paper proposes a Self-supervised Group Meiosis Contrastive learning framework (SGMC) based on the stimuli consistent EEG signals in human being. In the SGMC, a novel genetics-inspired data augmentation method, named Meiosis, is developed. It takes advantage of the alignment of stimuli among the EEG samples in a group for generating augmented groups by pairing, cross exchanging, and separating. And the model adopts a group projector to extract group-level feature representations from group EEG samples triggered by the same emotion video stimuli. Then contrastive learning is employed to maximize the similarity of group-level representations of augmented groups with the same stimuli. The SGMC achieves the state-of-the-art emotion recognition results on the publicly available DEAP dataset with an accuracy of 94.72% and 95.68% in valence and arousal dimensions, and also reaches competitive performance on the public SEED dataset with an accuracy of 94.04%. It is worthy of noting that the SGMC shows significant performance even when using limited labels. Moreover, the results of feature visualization suggest that the model might have learned video-level emotion-related feature representations to improve emotion recognition. And the effects of group size are further evaluated in the hyper parametric analysis. Finally, a control experiment and ablation study are carried out to examine the rationality of architecture. The code is provided publicly online.

研究动机与目标

• 为解决情绪识别中EEG标注数据有限的挑战，利用自监督学习。
• 利用多个受试者EEG样本间的刺激一致性，引导对比学习，而无需依赖人工标注。
• 开发一种受生物学减数分裂启发的新型数据增强方法，以保留与刺激相关的特征。
• 通过专用的群体投影器和对比损失，提升群体层级表征学习。
• 验证该框架在DEAP和SEED等公开EEG情绪识别数据集上的鲁棒性与泛化能力。

提出的方法

• 提出Meiosis方法，一种受遗传学启发的数据增强技术，通过配对、交叉交换和分离来自同一刺激组的EEG样本，生成增强后的群体。
• 采用群体投影器，从由同一情绪视频触发的多个EEG样本中提取共享的高层表征。
• 利用对比学习，最大化具有相同刺激的增强群体的群体层级表征之间的相似性。
• 基于刺激一致性设计正负样本对，引导模型学习与情绪相关的特征。
• 应用对称损失函数，确保训练稳定并实现对增强视图的不变表征学习。
• 采用两阶段流水线训练基础编码器和群体投影器：首先通过自监督对比学习进行预训练，随后使用少量标签进行微调。

Figure 1: Illustration of the proposed SGMC. During the process of pre-training, each group of samples is sampled from EEG samples corresponding to the same video clip stimuli. Then each group of EEG samples is augmented by genetics inspired Meiosis data augmentation to generate two augmented group

实验结果

研究问题

• RQ1 能否有效利用来自多个受试者的EEG样本之间的刺激一致性，以提升自监督情绪识别性能？
• RQ2 所提出的Meiosis数据增强方法与标准增强技术（如Mixup）相比，在保留与情绪相关特征方面表现如何？
• RQ3 在对比EEG情绪识别中，群体层级表征学习相较于个体样本学习的贡献有多大？
• RQ4 在低标签设置下，该框架表现如何？是否优于完全监督的基线方法？
• RQ5 通过特征可视化，模型在多大程度上学习到了视频层级的情绪表征？

主要发现

• SGMC在DEAP数据集上实现了最先进性能，效价准确率达到94.72%，唤醒度达到95.68%，超越了完全监督的基线方法。
• 在SEED数据集上，SGMC达到94.04%的准确率，即使在标注数据有限的情况下也表现出竞争力。
• 消融实验表明，移除Meiosis增强方法后，DEAP数据集准确率下降超过3%，SEED数据集下降1.2%，证实了其关键作用。
• 移除群体层级学习的Non-group变体在两个数据集上的性能均下降超过1.5%，证明了群体表征的重要性。
• 仅保留刺激一致性的Consistent-only变体（不包含群体样本或Meiosis）在DEAP上仍比完全监督基线高出1.7%，在SEED上高出0.6%，表明该一致性具有可利用价值。
• 特征可视化表明，模型学习到了视频层级的情绪相关表征，支持其在受试者间具有良好的泛化能力。

Figure 2: The illustration of sampling for a minibatch. Sampler first samples $P$ video clip and $2Q$ subjects. For each sampled video clip, next the sampler samples a group of EEG signals recorded when sampled $2Q$ subject watched it. Then $P$ groups of EEG samples are obtained for a minibatch.