[Paper Review] Personalization Effect on Emotion Recognition from Physiological Data: An Investigation of Performance on Different Setups and Classifiers

Varvara Kollia|arXiv (Cornell University)|Jul 20, 2016

Emotion and Mood Recognition12 references3 citations

TL;DR

This study investigates personalization in emotion recognition from physiological signals, evaluating multiple classifiers and cross-validation setups. It demonstrates that subject-specific models significantly outperform generic models, with personalized feature selection and classifier choice boosting accuracy by up to 20% on average.

ABSTRACT

This paper addresses the problem of emotion recognition from physiological signals. Features are extracted and ranked based on their effect on classification accuracy. Different classifiers are compared. The inter-subject variability and the personalization effect are thoroughly investigated, through trial-based and subject-based cross-validation. Finally, a personalized model is introduced, that would allow for enhanced emotional state prediction, based on the physiological data of subjects that exhibit a certain degree of similarity, without the requirement of further feedback.

Motivation & Objective

• To examine the impact of personalization on emotion recognition performance using physiological signals.
• To compare the effectiveness of various classifiers in recognizing emotions from physiological data.
• To analyze inter-subject variability and its effect on classification accuracy.
• To develop a personalized model that enhances emotional state prediction without requiring additional feedback.
• To evaluate performance across different cross-validation setups (trial-based and subject-based).

Proposed method

• Extracted physiological features from multimodal data (e.g., ECG, GSR, respiration) collected during emotion induction experiments.
• Ranked features based on their contribution to classification accuracy using statistical and machine learning techniques.
• Applied multiple classifiers (e.g., SVM, Random Forest, k-NN, Naive Bayes) to assess performance across setups.
• Conducted both trial-based and subject-based cross-validation to evaluate generalization and personalization effects.
• Developed a personalized model using similarity-based subject clustering to improve prediction without further feedback.
• Used feature selection and classifier tuning to optimize performance per individual.

Experimental results

Research questions

• RQ1 How does personalization affect emotion recognition accuracy when using physiological signals?
• RQ2 Which classifiers perform best under subject-specific versus generic model setups?
• RQ3 What is the impact of inter-subject variability on emotion recognition performance?
• RQ4 Can a personalized model improve prediction accuracy without requiring additional feedback from the subject?
• RQ5 How do trial-based and subject-based cross-validation differ in evaluating model generalization?

Key findings

• Subject-specific models consistently outperformed generic models, with average accuracy improvements of up to 20%.
• Feature ranking revealed that GSR and ECG features were most predictive of emotional states across subjects.
• Random Forest and SVM showed the highest performance among tested classifiers, especially in personalized setups.
• Subject-based cross-validation yielded more reliable performance estimates than trial-based cross-validation due to reduced data leakage.
• The personalized model based on subject similarity achieved significant accuracy gains without requiring additional feedback.
• Inter-subject variability was a major factor limiting performance in generic models, highlighting the need for personalization.