[Paper Review] Continuous Emotion Recognition during Music Listening Using EEG Signals: A Fuzzy Parallel Cascades Model

Fatemeh Hasanzadeh, Mohsen Annabestani+1 more|arXiv (Cornell University)|Oct 19, 2019

EEG and Brain-Computer Interfaces52 references37 citations

TL;DR

Introduces a fuzzy parallel cascades (FPC) model to predict continuous valence/arousal from time-varying EEG during music listening, outperforming LR, SVR, and LSTM baselines. Analyzed EEG from 15 subjects across 7 excerpts with frontal theta activity linked to valence.

ABSTRACT

A controversial issue in artificial intelligence is human emotion recognition. This paper presents a fuzzy parallel cascades (FPC) model for predicting the continuous subjective appraisal of the emotional content of music by time-varying spectral content of EEG signals. The EEG, along with an emotional appraisal of 15 subjects, was recorded during listening to seven musical excerpts. The emotional appraisement was recorded along the valence and arousal emotional axes as a continuous signal. The FPC model was composed of parallel cascades with each cascade containing a fuzzy logic-based system. The FPC model performance was evaluated by comparing with linear regression (LR), support vector regression (SVR) and Long Short Term Memory recurrent neural network (LSTM RNN) models. The RMSE of the FPC was lower than other models for the estimation of both valence and arousal of all musical excerpts. The lowest RMSE was 0.089 which was obtained in estimation of the valence of MS4 by the FPC model. The analysis of MI of frontal EEG with the valence confirms the role of frontal channels in theta frequency band in emotion recognition. Considering the dynamic variations of musical features during songs, employing a modeling approach to predict dynamic variations of the emotional appraisal can be a plausible substitute for the classification of musical excerpts into predefined labels.

Motivation & Objective

• Motivate continuous, dynamic emotion appraisal prediction rather than fixed-label classification during music listening.
• Develop a fuzzy parallel cascades (FPC) framework where each cascade uses fuzzy logic to map EEG features to emotional appraisals.
• Evaluate FPC against linear regression, support vector regression, and LSTM on time-varying valence and arousal data.
• Investigate the neurophysiological correlates, notably frontal theta-band activity, in emotion estimation from EEG.

Proposed method

• Construct time-varying EEG features captured while subjects listened to seven musical excerpts.
• Design a parallel cascades architecture where each cascade consists of a fuzzy logic-based system.
• Train and compare FPC against LR, SVR, and LSTM on continuous valence and arousal prediction.
• Use RMSE as the evaluation metric for both valence and arousal estimations.
• Analyze mutual information (MI) between frontal EEG channels and valence to identify neurophysiological relevance.

Experimental results

Research questions

• RQ1 Can a fuzzy parallel cascades model improve continuous estimation of valence and arousal from EEG during music listening compared to baseline models?
• RQ2 What is the relative RMSE performance of FPC versus LR, SVR, and LSTM across multiple musical excerpts?
• RQ3 Which EEG features, particularly frontal channels in theta band, are most informative for continuous emotion appraisal?
• RQ4 Does modeling dynamic emotional variation yield better approximation than single-label classification for musical excerpts?

Key findings

• FPC achieves lower RMSE than LR, SVR, and LSTM across all musical excerpts for both valence and arousal.
• The best single result reported is RMSE = 0.089 for valence estimation of excerpt MS4 using FPC.
• Frontal EEG channels in the theta band show meaningful MI with valence, supporting frontal theta involvement in emotion recognition.
• Modeling dynamic emotional variation aligns with continuous appraisal rather than fixed excerpt classifications.