[Paper Review] Automatic Emotion Recognition (AER) System based on Two-Level Ensemble of Lightweight Deep CNN Models

Emad-ul-Haq Qazi, Muhammad Hussain+2 more|arXiv (Cornell University)|Apr 30, 2019

EEG and Brain-Computer Interfaces43 references11 citations

TL;DR

This paper proposes Deep-AER, a two-level ensemble of lightweight 1D-CNN models for automatic emotion recognition (AER) from EEG signals. By leveraging a pyramidal 1D-CNN with few parameters and combining predictions via majority voting across channels and levels, the system achieves 98.43% accuracy on HV vs LV and 97.65% on HA vs LA classification, outperforming state-of-the-art methods on the DEAP dataset.

ABSTRACT

Emotions play a crucial role in human interaction, health care and security investigations and monitoring. Automatic emotion recognition (AER) using electroencephalogram (EEG) signals is an effective method for decoding the real emotions, which are independent of body gestures, but it is a challenging problem. Several automatic emotion recognition systems have been proposed, which are based on traditional hand-engineered approaches and their performances are very poor. Motivated by the outstanding performance of deep learning (DL) in many recognition tasks, we introduce an AER system (Deep-AER) based on EEG brain signals using DL. A DL model involves a large number of learnable parameters, and its training needs a large dataset of EEG signals, which is difficult to acquire for AER problem. To overcome this problem, we proposed a lightweight pyramidal one-dimensional convolutional neural network (LP-1D-CNN) model, which involves a small number of learnable parameters. Using LP-1D-CNN, we build a two level ensemble model. In the first level of the ensemble, each channel is scanned incrementally by LP-1D-CNN to generate predictions, which are fused using majority vote. The second level of the ensemble combines the predictions of all channels of an EEG signal using majority vote for detecting the emotion state. We validated the effectiveness and robustness of Deep-AER using DEAP, a benchmark dataset for emotion recognition research. The results indicate that FRONT plays dominant role in AER and over this region, Deep-AER achieved the accuracies of 98.43% and 97.65% for two AER problems, i.e., high valence vs low valence (HV vs LV) and high arousal vs low arousal (HA vs LA), respectively. The comparison reveals that Deep-AER outperforms the state-of-the-art systems with large margin. The Deep-AER system will be helpful in monitoring for health care and security investigations.

Motivation & Objective

• To address the challenge of low performance in traditional hand-engineered AER systems using EEG signals.
• To overcome the data scarcity issue in EEG-based AER by designing a lightweight deep learning model with few learnable parameters.
• To improve emotion recognition accuracy through a two-level ensemble strategy combining channel-wise and signal-wide predictions.
• To validate the model's robustness and effectiveness on the benchmark DEAP dataset.

Proposed method

• Proposed a lightweight pyramidal one-dimensional convolutional neural network (LP-1D-CNN) with minimal learnable parameters to reduce data dependency.
• Applied the LP-1D-CNN independently to each EEG channel in the first-level ensemble, generating individual predictions.
• Fused first-level predictions using majority voting to form channel-level emotion estimates.
• Combined all channel-level predictions in a second-level ensemble using majority voting to determine the final emotion state.
• Trained and evaluated the model on the DEAP dataset, using high valence vs low valence (HV vs LV) and high arousal vs low arousal (HA vs LA) as classification tasks.
• Utilized EEG signals from frontal brain regions as the primary source of emotion-relevant features, given their dominant role in emotion processing.

Experimental results

Research questions

• RQ1 Can a lightweight deep learning model achieve high accuracy in EEG-based emotion recognition with limited training data?
• RQ2 Does a two-level ensemble strategy improve emotion recognition performance by combining predictions across EEG channels and levels?
• RQ3 Which EEG regions contribute most significantly to accurate emotion recognition in AER systems?
• RQ4 How does the proposed Deep-AER system compare to state-of-the-art methods in terms of accuracy and robustness on benchmark datasets?
• RQ5 Can majority voting across multiple channel-specific predictions enhance the reliability of emotion classification?

Key findings

• The Deep-AER system achieved 98.43% accuracy in distinguishing high valence from low valence emotions on the DEAP dataset.
• The system attained 97.65% accuracy in classifying high arousal versus low arousal states, demonstrating strong performance on the second emotion dimension.
• Frontal EEG regions were found to play a dominant role in emotion recognition, supporting the focus on this area in the model design.
• The two-level ensemble strategy significantly improved classification robustness by aggregating predictions across multiple EEG channels.
• Deep-AER outperformed existing state-of-the-art AER systems by a large margin, confirming the effectiveness of the lightweight and ensemble-based approach.
• The LP-1D-CNN model's low parameter count makes it suitable for deployment in resource-constrained environments, such as real-time health monitoring systems.