[Paper Review] Decoding Speech Envelopes from Electroencephalogram with a Contrastive Pearson Correlation Coefficient Loss

Yayun Liang, Yuanming Zhang+3 more|arXiv (Cornell University)|Jan 28, 2026

EEG and Brain-Computer Interfaces0 citations

TL;DR

The paper introduces a contrastive Pearson correlation coefficient loss to explicitly maximize the difference between attended and unattended envelope correlations, improving EEG-based auditory attention decoding across multiple models and datasets.

ABSTRACT

Recent advances in reconstructing speech envelopes from Electroencephalogram (EEG) signals have enabled continuous auditory attention decoding (AAD) in multi-speaker environments. Most Deep Neural Network (DNN)-based envelope reconstruction models are trained to maximize the Pearson correlation coefficients (PCC) between the attended envelope and the reconstructed envelope (attended PCC). While the difference between the attended PCC and the unattended PCC plays an essential role in auditory attention decoding, existing methods often focus on maximizing the attended PCC. We therefore propose a contrastive PCC loss which represents the difference between the attended PCC and the unattended PCC. The proposed approach is evaluated on three public EEG AAD datasets using four DNN architectures. Across many settings, the proposed objective improves envelope separability and AAD accuracy, while also revealing dataset- and architecture-dependent failure cases.

Motivation & Objective

• Motivate improved separation between attended and unattended speech envelopes in EEG-based decoding.
• Propose a contrastive loss that combines attended and unattended PCCs to enhance decoding performance.
• Evaluate the proposed loss across multiple DNN architectures and three public EEG datasets.
• Analyze how the PCC difference relates to decoding accuracy and discuss limitations across datasets and models.

Proposed method

• Model EEG-based envelope reconstruction as a regression problem.
• Define attended PCC and unattended PCCs with respect to multiple concurrent speakers.
• Introduce L_DeltaPCC = -rho_a + (1/(N_speaker-1)) * sum(rho_u,j) to promote attended-unattended separation.
• Train four architectures (VLAAI, LSM, EEGMamba, EEGDeformer) with PCC loss and DeltaPCC loss.
• Use fourfold leave-one-trial-out cross-validation and AdamW optimization.
• Preprocess EEG (1–32 Hz, 128 Hz), extract broadband envelopes from ERB gammatone subbands, align data.

Experimental results

Research questions

• RQ1 Does the proposed contrastive DeltaPCC loss improve decoding accuracy compared to the standard PCC loss across different architectures?
• RQ2 How does DeltaPCC affect the difference between attended and unattended PCCs, and how does this relate to AAD performance?
• RQ3 Is the improvement consistent across datasets (KUL, DTU, KUL-AV-GC) and window lengths?
• RQ4 What are the failure cases and data/architecture factors that influence the effectiveness of the DeltaPCC loss?

Key findings

• DeltaPCC training generally yields higher decoding accuracy than PCC training across models and datasets.
• DeltaPCC increases the attended-unattended PCC difference, with an average relative improvement of 17.84% across conditions.
• Decoding accuracy correlates more strongly with DeltaPCC than with attended PCC alone (R^2 > 0.5 for DeltaPCC).
• Some models/datasets show no improvement or degrade with DeltaPCC, indicating sensitivity to data characteristics and window length.
• Improvement is more pronounced with longer windows (e.g., 10 s) and varies by dataset and model.