[Paper Review] Affective State Recognition through EEG Signals Feature Level Fusion and Ensemble Classifier

Md. Mahbubur Rahman, Akash Poddar+2 more|arXiv (Cornell University)|Feb 14, 2021

Emotion and Mood Recognition11 references5 citations

TL;DR

This study proposes a novel affective state recognition framework using EEG signals through feature-level fusion and an ensemble classifier. It achieves 89.06% accuracy in classifying four basic affective states—happy, sad, disgusted, and peaceful—by fusing statistical and advanced spectral features from EEG data collected during emotional video stimulation, with random forest outperforming other classifiers in the ensemble.

ABSTRACT

Human affects are complex paradox and an active research domain in affective computing. Affects are traditionally determined through a self-report based psychometric questionnaire or through facial expression recognition. However, few state-of-the-arts pieces of research have shown the possibilities of recognizing human affects from psychophysiological and neurological signals. In this article, electroencephalogram (EEG) signals are used to recognize human affects. The electroencephalogram (EEG) of 100 participants are collected where they are given to watch one-minute video stimuli to induce different affective states. The videos with emotional tags have a variety range of affects including happy, sad, disgust, and peaceful. The experimental stimuli are collected and analyzed intensively. The interrelationship between the EEG signal frequencies and the ratings given by the participants are taken into consideration for classifying affective states. Advanced feature extraction techniques are applied along with the statistical features to prepare a fused feature vector of affective state recognition. Factor analysis methods are also applied to select discriminative features. Finally, several popular supervised machine learning classifier is applied to recognize different affective states from the discriminative feature vector. Based on the experiment, the designed random forest classifier produces 89.06% accuracy in classifying four basic affective states.

Motivation & Objective

• To develop a robust affective state recognition system using non-invasive EEG signals.
• To address the limitations of self-report and facial expression methods by leveraging neurophysiological signals.
• To improve classification accuracy through advanced feature fusion and ensemble learning techniques.
• To identify discriminative EEG features correlated with subjective emotional ratings.

Proposed method

• EEG signals were collected from 100 participants while viewing one-minute emotionally tagged video stimuli.
• Statistical and spectral features (e.g., power in delta, theta, alpha, beta, gamma bands) were extracted from the EEG signals.
• Feature-level fusion combined multiple feature types into a single discriminative feature vector.
• Factor analysis was applied to reduce dimensionality and select the most informative features.
• Multiple supervised machine learning classifiers were evaluated, with random forest selected as the final ensemble model.
• Affective states were classified based on the fused feature vector using the trained ensemble classifier.

Experimental results

Research questions

• RQ1 Can EEG signals reliably capture and differentiate between basic affective states such as happiness, sadness, disgust, and peace?
• RQ2 How effective is feature-level fusion of statistical and spectral EEG features in improving affective state classification accuracy?
• RQ3 Which ensemble classifier performs best in recognizing multiple affective states from EEG data?
• RQ4 To what extent do interrelationships between EEG frequency bands and subjective emotional ratings enhance classification performance?

Key findings

• The proposed feature-level fusion approach significantly improved classification performance over individual feature sets.
• The random forest classifier achieved the highest accuracy of 89.06% in distinguishing four basic affective states.
• Factor analysis successfully reduced feature dimensionality while preserving discriminative information.
• Spectral power in the alpha and beta bands showed strong correlations with subjective emotional ratings.
• The ensemble model demonstrated robustness and generalization across diverse affective states.
• The study confirms the feasibility of using EEG signals for real-time, non-invasive affective state recognition in human-computer interaction.