[Paper Review] A Lightweight Domain Adversarial Neural Network Based on Knowledge Distillation for EEG-based Cross-subject Emotion Recognition

Zhe Wang, Yongxiong Wang+3 more|arXiv (Cornell University)|May 12, 2023

EEG and Brain-Computer Interfaces4 citations

TL;DR

This paper proposes a lightweight domain adversarial neural network enhanced by knowledge distillation for EEG-based cross-subject emotion recognition. A transformer-based teacher model guides a Bi-LSTM student model to learn domain-invariant features, improving performance on the DEAP dataset with subject-independent evaluation, achieving state-of-the-art results in arousal and valence classification.

ABSTRACT

Individual differences of Electroencephalogram (EEG) could cause the domain shift which would significantly degrade the performance of cross-subject strategy. The domain adversarial neural networks (DANN), where the classification loss and domain loss jointly update the parameters of feature extractor, are adopted to deal with the domain shift. However, limited EEG data quantity and strong individual difference are challenges for the DANN with cumbersome feature extractor. In this work, we propose knowledge distillation (KD) based lightweight DANN to enhance cross-subject EEG-based emotion recognition. Specifically, the teacher model with strong context learning ability is utilized to learn complex temporal dynamics and spatial correlations of EEG, and robust lightweight student model is guided by the teacher model to learn more difficult domain-invariant features. In the feature-based KD framework, a transformer-based hierarchical temporalspatial learning model is served as the teacher model. The student model, which is composed of Bi-LSTM units, is a lightweight version of the teacher model. Hence, the student model could be supervised to mimic the robust feature representations of teacher model by leveraging complementary latent temporal features and spatial features. In the DANN-based cross-subject emotion recognition, we combine the obtained student model and a lightweight temporal-spatial feature interaction module as the feature extractor. And the feature aggregation is fed to the emotion classifier and domain classifier for domain-invariant feature learning. To verify the effectiveness of the proposed method, we conduct the subject-independent experiments on the public dataset DEAP with arousal and valence classification. The outstanding performance and t-SNE visualization of latent features verify the advantage and effectiveness of the proposed method.

Motivation & Objective

• Address domain shift in EEG-based emotion recognition due to individual differences and limited data.
• Overcome the limitations of heavy feature extractors in domain adversarial networks under low-data regimes.
• Develop a lightweight yet robust model for cross-subject EEG emotion recognition using knowledge distillation.
• Enable effective domain-invariant feature learning by transferring complex spatiotemporal representations from a strong teacher to a compact student.
• Improve generalization across subjects using a distilled, efficient student model trained on temporal-spatial dynamics.

Proposed method

• Employ a transformer-based hierarchical temporal-spatial model as the teacher to capture complex EEG dynamics and spatial correlations.
• Design a lightweight Bi-LSTM-based student model to mimic the teacher’s latent representations through knowledge distillation.
• Implement a feature-based distillation framework where student learns from both temporal and spatial features of the teacher.
• Integrate the distilled student model with a lightweight temporal-spatial feature interaction module as the final feature extractor.
• Train the system end-to-end using domain adversarial learning, jointly optimizing classification and domain confusion losses.
• Use adversarial training to align feature distributions across subjects, promoting domain-invariant representation learning.

Experimental results

Research questions

• RQ1 Can knowledge distillation effectively transfer complex spatiotemporal EEG representations from a heavy teacher model to a lightweight student model?
• RQ2 Does the distilled student model achieve better cross-subject generalization than standard DANN under limited EEG data?
• RQ3 How effective is the proposed method in learning domain-invariant features across subjects in EEG-based emotion recognition?
• RQ4 Can a lightweight student model trained via distillation outperform larger models in subject-independent evaluation?
• RQ5 What is the impact of the temporal-spatial feature interaction module on representation quality and classification accuracy?

Key findings

• The proposed method achieves state-of-the-art performance on the DEAP dataset for both arousal and valence classification in subject-independent settings.
• t-SNE visualization confirms that the latent features learned by the student model are more domain-invariant and better separated across subjects.
• The knowledge distillation framework enables the lightweight student model to effectively learn complex temporal dynamics and spatial correlations from the teacher.
• The combination of distillation and domain adversarial training significantly improves generalization across subjects despite limited training data.
• The student model achieves high performance with reduced parameters and inference cost, making it suitable for real-time applications.
• Quantitative results show improved accuracy and F1-scores over baseline DANN and other state-of-the-art methods on the DEAP dataset.