[Paper Review] CLDTA: Contrastive Learning based on Diagonal Transformer Autoencoder for Cross-Dataset EEG Emotion Recognition

Yuan Liao, Yuhong Zhang+6 more|arXiv (Cornell University)|Jun 12, 2024

EEG and Brain-Computer Interfaces5 citations

TL;DR

Introduces CLDTA, a contrastive learning framework with a diagonal masking transformer autoencoder to enhance cross-dataset EEG emotion recognition and rapid subject adaptation.

ABSTRACT

Recent advances in non-invasive EEG technology have broadened its application in emotion recognition, yielding a multitude of related datasets. Yet, deep learning models struggle to generalize across these datasets due to variations in acquisition equipment and emotional stimulus materials. To address the pressing need for a universal model that fluidly accommodates diverse EEG dataset formats and bridges the gap between laboratory and real-world data, we introduce a novel deep learning framework: the Contrastive Learning based Diagonal Transformer Autoencoder (CLDTA), tailored for EEG-based emotion recognition. The CLDTA employs a diagonal masking strategy within its encoder to extracts full-channel EEG data's brain network knowledge, facilitating transferability to the datasets with fewer channels. And an information separation mechanism improves model interpretability by enabling straightforward visualization of brain networks. The CLDTA framework employs contrastive learning to distill subject-independent emotional representations and uses a calibration prediction process to enable rapid adaptation of the model to new subjects with minimal samples, achieving accurate emotion recognition. Our analysis across the SEED, SEED-IV, SEED-V, and DEAP datasets highlights CLDTA's consistent performance and proficiency in detecting both task-specific and general features of EEG signals related to emotions, underscoring its potential to revolutionize emotion recognition research.

Motivation & Objective

• Address cross-dataset generalization in EEG emotion recognition due to varying equipment and stimuli.
• Propose a diagonal masking encoder to capture full-channel brain network knowledge and enable transfer to datasets with fewer channels.
• Incorporate contrastive learning to distill subject-independent emotional representations.
• Enable rapid model adaptation to new subjects with minimal labeled samples.

Proposed method

• Use a diagonal masking strategy in the encoder to extract brain-network information from full-channel EEG data.
• Employ a transformer autoencoder architecture to model EEG signals with diagonal masking.
• Integrate a contrastive learning objective to learn subject-invariant emotional representations.
• Introduce an information separation mechanism to improve interpretability and visualize brain networks.
• Implement a calibration/prediction process to adapt the model to new subjects with few samples.

Experimental results

Research questions

• RQ1 Can CLDTA achieve cross-dataset generalization across SEED, SEED-IV, SEED-V, and DEAP datasets?
• RQ2 Does the diagonal masking encoder effectively transfer knowledge to datasets with fewer channels?
• RQ3 Can contrastive learning yield robust, subject-independent emotional representations for EEG signals?
• RQ4 Does the information separation mechanism enhance interpretability of brain networks?
• RQ5 How rapidly can the model adapt to new subjects with minimal labeled data?

Key findings

• CLDTA demonstrates consistent performance across multiple EEG emotion datasets.
• The diagonal masking encoder captures brain-network information enabling transfer to datasets with fewer channels.
• Contrastive learning yields subject-invariant representations for emotion-related EEG signals.
• An information separation mechanism facilitates straightforward visualization of brain networks.
• A calibration/prediction process enables rapid adaptation to new subjects with few labeled samples.