[Paper Review] Study on Feature Subspace of Archetypal Emotions for Speech Emotion Recognition

Xi Ma, Zhiyong Wu+4 more|arXiv (Cornell University)|Nov 17, 2016

Emotion and Mood Recognition10 references3 citations

TL;DR

This paper proposes a divide-and-conquer approach for speech emotion recognition by constructing emotion-pair-specific feature subspaces, training binary classifiers on each, and combining results via voting. The method achieves significantly higher accuracy than traditional multi-classification using a global feature space, demonstrating that emotion-specific subspaces better capture discriminative features for distinct emotion pairs.

ABSTRACT

Feature subspace selection is an important part in speech emotion recognition. Most of the studies are devoted to finding a feature subspace for representing all emotions. However, some studies have indicated that the features associated with different emotions are not exactly the same. Hence, traditional methods may fail to distinguish some of the emotions with just one global feature subspace. In this work, we propose a new divide and conquer idea to solve the problem. First, the feature subspaces are constructed for all the combinations of every two different emotions (emotion-pair). Bi-classifiers are then trained on these feature subspaces respectively. The final emotion recognition result is derived by the voting and competition method. Experimental results demonstrate that the proposed method can get better results than the traditional multi-classification method.

Motivation & Objective

• To address the limitation of global feature subspaces in distinguishing emotions with distinct acoustic characteristics.
• To investigate whether emotion-specific feature subspaces can improve classification performance in speech emotion recognition.
• To develop a scalable, divide-and-conquer framework that treats multi-class emotion recognition as a set of binary classification problems.
• To validate the effectiveness of this approach across different feature selection and transformation techniques.

Proposed method

• Construct feature subspaces tailored for each pair of archetypal emotions (e.g., Anger vs. Happiness), rather than using a single global subspace.
• Apply feature selection (e.g., Genetic Algorithm) or feature space transformation (e.g., Neural Networks) to identify the most discriminative features for each emotion-pair.
• Train a binary classifier (e.g., Logistic Regression, SVM, Neural Network) on each emotion-pair-specific feature subspace.
• Combine individual binary decisions through a voting and competition mechanism to determine the final emotion label.
• Ensure all subspaces contain 50 features for fair comparison with the global feature space.
• Use utterance-level low-level acoustic descriptors (e.g., MFCC, F0, energy, zero-crossing rate) with delta coefficients and statistical functionals.

Experimental results

Research questions

• RQ1 Can emotion-pair-specific feature subspaces improve speech emotion recognition performance compared to a single global feature subspace?
• RQ2 How does the performance of the proposed method compare to traditional multi-classification using the same global feature space?
• RQ3 Does the divide-and-conquer strategy remain effective when using feature space transformation instead of feature subset selection?
• RQ4 To what extent do the feature subspaces for different emotion-pairs overlap with the global feature subspace?
• RQ5 Can the method generalize across different classifiers and feature engineering techniques?

Key findings

• The proposed 'Bi-classification and voting' method achieved a recognition accuracy of 0.735 using Logistic Regression, significantly higher than the 0.653 accuracy of the traditional multi-classification method (p < 0.05).
• With SVM, the proposed method reached 0.625 accuracy, compared to 0.513 for the multi-classification baseline, confirming a statistically significant improvement.
• The method outperformed the global feature space baseline across all emotions, including Anger, Boredom, Happiness, Sadness, and Fear, with consistent gains in recognition rate.
• Only 5 or fewer features were shared between any emotion-pair-specific subspace and the global feature subspace, indicating substantial structural differences and validating the need for pair-specific subspaces.
• The method remained effective when using neural network-based feature space transformation, achieving 0.652 accuracy versus 0.552 for the global method, confirming its generality across feature engineering strategies.