[Paper Review] ScalingNet: extracting features from raw EEG data for emotion recognition

Jingzhao Hu, Chen Wang+3 more|arXiv (Cornell University)|Feb 7, 2021

EEG and Brain-Computer Interfaces25 references4 citations

TL;DR

This paper proposes ScalingNet, a novel end-to-end deep learning architecture that uses a data-driven scaling layer to automatically extract spectrogram-like features directly from raw EEG signals, eliminating the need for handcrafted features. The method achieves state-of-the-art performance on the DEAP benchmark, with 69.99% accuracy for arousal, 71.13% for valence, and 70.78% for dominance, demonstrating superior robustness and generalization without manual feature engineering.

ABSTRACT

Convolutional Neural Networks(CNNs) has achieved remarkable performance breakthrough in a variety of tasks. Recently, CNNs based methods that are fed with hand-extracted EEG features gradually produce a powerful performance on the EEG data based emotion recognition task. In this paper, we propose a novel convolutional layer allowing to adaptively extract effective data-driven spectrogram-like features from raw EEG signals, which we reference as scaling layer. Further, it leverages convolutional kernels scaled from one data-driven pattern to exposed a frequency-like dimension to address the shortcomings of prior methods requiring hand-extracted features or their approximations. The proposed neural network architecture based on the scaling layer, references as ScalingNet, has achieved the state-of-the-art result across the established DEAP benchmark dataset.

Motivation & Objective

• To address the limitations of hand-extracted EEG features, which rely on strong assumptions and are task-specific.
• To develop an end-to-end deep learning framework that automatically learns robust, data-driven representations from raw EEG signals.
• To eliminate the need for data alignment and feature engineering in EEG-based emotion recognition.
• To design a novel convolutional layer that dynamically generates scaled kernels to extract frequency-like representations.

Proposed method

• Introduces the scaling layer, which generates a series of convolutional kernels by scaling from a single data-driven pattern.
• The scaling layer produces a spectrogram-like feature map by applying these scaled kernels to raw EEG signals, capturing both time and frequency-like information.
• The architecture is fully convolutional and compatible with existing deep learning models, enabling end-to-end training.
• The method is designed to handle variable-length EEG signals without requiring alignment or segmentation.
• The scaling layer uses a learnable base kernel that is adaptively scaled across multiple levels to simulate frequency resolution.
• The model is trained end-to-end on raw EEG data using standard backpropagation and optimization.

Experimental results

Research questions

• RQ1 Can a data-driven convolutional layer extract robust, spectrogram-like features from raw EEG signals without handcrafted features?
• RQ2 How does the performance of the scaling layer compare to standard convolutional layers in EEG-based emotion recognition?
• RQ3 What is the optimal model capacity (kernel length) for the scaling layer in terms of representational power and accuracy?
• RQ4 How interpretable are the feature maps generated by the scaling layer in terms of frequency-like patterns?
• RQ5 Does the proposed end-to-end architecture outperform existing state-of-the-art methods on standard EEG emotion recognition benchmarks?

Key findings

• ScalingNet achieved state-of-the-art performance on the DEAP benchmark, with 69.99% accuracy for arousal, 71.13% for valence, and 70.78% for dominance.
• The scaling layer outperformed standard convolutional layers, with a 4.25% absolute improvement in arousal accuracy (69.99% vs. 65.74%).
• The optimal model capacity was found to be a base kernel length of 33, which maximized representational capacity and recognition accuracy.
• Visualizations of the feature maps revealed distinct low- and high-frequency-like energy patterns, indicating the layer learns meaningful spectral representations.
• Ablation studies confirmed that the scaling layer significantly enhances feature extraction and generalization, proving its critical role in performance.