[论文解读] A Lightweight Domain Adversarial Neural Network Based on Knowledge Distillation for EEG-based Cross-subject Emotion Recognition

Zhe Wang, Yongxiong Wang+3 more|arXiv (Cornell University)|May 12, 2023

EEG and Brain-Computer Interfaces被引用 4

一句话总结

本论文提出了一种通过知识蒸馏增强的轻量化领域对抗神经网络，用于基于EEG的跨被试情绪识别。基于Transformer的教师模型指导Bi-LSTM学生模型学习领域不变特征，在DEAP数据集上采用被试独立评估，实现了在唤醒度和效价分类任务上的最先进性能。

ABSTRACT

Individual differences of Electroencephalogram (EEG) could cause the domain shift which would significantly degrade the performance of cross-subject strategy. The domain adversarial neural networks (DANN), where the classification loss and domain loss jointly update the parameters of feature extractor, are adopted to deal with the domain shift. However, limited EEG data quantity and strong individual difference are challenges for the DANN with cumbersome feature extractor. In this work, we propose knowledge distillation (KD) based lightweight DANN to enhance cross-subject EEG-based emotion recognition. Specifically, the teacher model with strong context learning ability is utilized to learn complex temporal dynamics and spatial correlations of EEG, and robust lightweight student model is guided by the teacher model to learn more difficult domain-invariant features. In the feature-based KD framework, a transformer-based hierarchical temporalspatial learning model is served as the teacher model. The student model, which is composed of Bi-LSTM units, is a lightweight version of the teacher model. Hence, the student model could be supervised to mimic the robust feature representations of teacher model by leveraging complementary latent temporal features and spatial features. In the DANN-based cross-subject emotion recognition, we combine the obtained student model and a lightweight temporal-spatial feature interaction module as the feature extractor. And the feature aggregation is fed to the emotion classifier and domain classifier for domain-invariant feature learning. To verify the effectiveness of the proposed method, we conduct the subject-independent experiments on the public dataset DEAP with arousal and valence classification. The outstanding performance and t-SNE visualization of latent features verify the advantage and effectiveness of the proposed method.

研究动机与目标

• 解决由于个体差异和数据有限导致的EEG情绪识别中的领域偏移问题。
• 克服在低数据环境下领域对抗网络中重型特征提取器的局限性。
• 通过知识蒸馏开发一种轻量化但鲁棒的模型，用于跨被试EEG情绪识别。
• 通过将强教师模型的复杂时空表征迁移至紧凑的学生模型，实现有效的领域不变特征学习。
• 通过在时间-空间动态上训练的蒸馏、高效的学生模型，提升跨被试的泛化能力。

提出的方法

• 采用基于Transformer的分层时空模型作为教师模型，以捕捉复杂的EEG动态和空间相关性。
• 设计一种轻量化的基于Bi-LSTM的学生模型，通过知识蒸馏模仿教师模型的潜在表征。
• 实施基于特征的蒸馏框架，使学生模型从教师模型的时间和空间特征中学习。
• 将蒸馏后的学生模型与轻量化的时空特征交互模块结合，作为最终的特征提取器。
• 采用端到端的领域对抗学习进行训练，联合优化分类损失和领域混淆损失。
• 通过对抗训练对齐不同被试之间的特征分布，促进领域不变表征学习。

实验结果

研究问题

• RQ1 知识蒸馏能否有效将复杂时空EEG表征从重型教师模型迁移至轻量化学生模型？
• RQ2 在EEG数据有限的情况下，蒸馏后的学生模型是否在跨被试泛化能力上优于标准DANN？
• RQ3 所提出方法在EEG情绪识别中学习跨被试领域不变特征方面的有效性如何？
• RQ4 通过蒸馏训练的轻量化学生模型是否能在被试独立评估中超越更大的模型？
• RQ5 时空特征交互模块对表征质量和分类准确率的影响如何？

主要发现

• 所提方法在DEAP数据集上实现了被试独立设置下唤醒度和效价分类的最先进性能。
• t-SNE可视化结果表明，学生模型学习到的潜在特征更具领域不变性，且在不同被试间分离得更好。
• 知识蒸馏框架使轻量化学生模型能够有效学习教师模型的复杂时间动态和空间相关性。
• 蒸馏与领域对抗训练的结合显著提升了在有限训练数据下的跨被试泛化能力。
• 学生模型在参数量和推理成本显著减少的情况下仍保持高性能，适用于实时应用。
• 定量结果表明，与基线DANN及其他最先进方法相比，该方法在DEAP数据集上的准确率和F1分数均有提升。