[Paper Review] EEGDnet: Fusing Non-Local and Local Self-Similarity for 1-D EEG Signal Denoising with 2-D Transformer

Peng Yi, Kecheng Chen+4 more|arXiv (Cornell University)|Sep 9, 2021

EEG and Brain-Computer Interfaces43 references7 citations

TL;DR

EEGDnet is a novel 1D EEG denoising model that leverages a 2D Transformer to jointly exploit non-local self-similarity in self-attention blocks and local self-similarity in feed-forward blocks, achieving state-of-the-art performance with reduced model complexity. It outperforms existing methods in both quantitative metrics and visual quality while maintaining low computational cost for embedded BCI applications.

ABSTRACT

Electroencephalogram (EEG) has shown a useful approach to produce a brain-computer interface (BCI). One-dimensional (1-D) EEG signal is yet easily disturbed by certain artifacts (a.k.a. noise) due to the high temporal resolution. Thus, it is crucial to remove the noise in received EEG signal. Recently, deep learning-based EEG signal denoising approaches have achieved impressive performance compared with traditional ones. It is well known that the characteristics of self-similarity (including non-local and local ones) of data (e.g., natural images and time-domain signals) are widely leveraged for denoising. However, existing deep learning-based EEG signal denoising methods ignore either the non-local self-similarity (e.g., 1-D convolutional neural network) or local one (e.g., fully connected network and recurrent neural network). To address this issue, we propose a novel 1-D EEG signal denoising network with 2-D transformer, namely EEGDnet. Specifically, we comprehensively take into account the non-local and local self-similarity of EEG signal through the transformer module. By fusing non-local self-similarity in self-attention blocks and local self-similarity in feed forward blocks, the negative impact caused by noises and outliers can be reduced significantly. Extensive experiments show that, compared with other state-of-the-art models, EEGDnet achieves much better performance in terms of both quantitative and qualitative metrics.

Motivation & Objective

• To address the limitation of existing deep learning-based EEG denoising methods that neglect either non-local or local self-similarity in 1D EEG signals.
• To develop a lightweight, end-to-end deep learning model suitable for low-power embedded BCI devices.
• To explore the effectiveness of 2D Transformer architecture in capturing both global and local signal patterns in 1D time-series EEG data.
• To balance model efficiency and denoising performance by optimizing architectural components such as segmentation and depth.

Proposed method

• Proposes a 2D Transformer-based architecture that transforms 1D EEG signals into 2D feature maps to enable joint modeling of non-local and local self-similarity.
• Utilizes self-attention mechanisms in the 2D Transformer to capture non-local self-similarity across spatial and temporal dimensions.
• Employs feed-forward networks with 1D local segmentation to model local self-similarity within small signal windows.
• Introduces a parameter-efficient design by controlling the segmentation size (k and q) to balance model complexity and performance.
• Stacks multiple Transformer blocks in an end-to-end fashion to learn hierarchical denoising representations.
• Optimizes model efficiency by minimizing parameters and FLOPs, targeting deployment on resource-constrained embedded BCI systems.

Experimental results

Research questions

• RQ1 Can fusing non-local and local self-similarity in a 2D Transformer architecture improve 1D EEG signal denoising performance compared to existing methods?
• RQ2 How does the segmentation size (k and q) in the 2D feature map affect the trade-off between model complexity and denoising accuracy?
• RQ3 Does the proposed EEGDnet model achieve superior performance in both quantitative metrics and visual quality compared to state-of-the-art models?
• RQ4 To what extent does the 2D Transformer architecture reduce model parameters and FLOPs while maintaining high denoising accuracy for embedded BCI deployment?

Key findings

• EEGDnet achieves the best performance across all quantitative metrics, including RRMSE (0.491), spectral RRMSE (0.626), and CC (0.732), outperforming all baselines.
• The model exhibits superior visual quality in denoising results, with waveforms closely matching the ground truth in both ocular and muscle artifact cases.
• Ablation studies show that optimal performance is achieved when the segmentation size (k) and local window size (q) are balanced, avoiding degradation to a fully connected network.
• The model achieves a significant reduction in parameters (182K) and FLOPs (1682M) compared to other deep learning models, making it suitable for low-power embedded systems.
• Reducing model depth and number of attention heads improves performance on the small dataset, suggesting that simpler architectures generalize better in this setting.
• EEGDnet demonstrates generalization potential beyond EEG denoising, with applicability to other 1D signal processing tasks.