[论文解读] Emotion Recognition Through Observer's Physiological Signals

Yang Liu, Tom Gedeon+3 more|arXiv (Cornell University)|Feb 19, 2020

EEG and Brain-Computer Interfaces参考文献 11被引用 4

一句话总结

本文介绍了PAFEW数据集，该数据集从24名观察者观看AFEW中的情绪诱导电影片段中收集生理信号，重点研究皮肤电活动（EDA）在情绪识别中的应用。采用三层神经网络基线模型，在单名参与者数据上达到42.08%的准确率；通过引入效价信息的两步网络结构，准确率提升了4.9%，表明在该数据集中，效价作为辅助信号比唤醒度更有效于情绪分类。

ABSTRACT

Emotion recognition based on physiological signals is a hot topic and has a wide range of applications, like safe driving, health care and creating a secure society. This paper introduces a physiological dataset PAFEW, which is obtained using movie clips from the Acted Facial Expressions in the Wild (AFEW) dataset as stimuli. To establish a baseline, we use the electrodermal activity (EDA) signals in this dataset and extract 6 features from each signal series corresponding to each movie clip to recognize 7 emotions, i.e., Anger, Disgust, Fear, Happy, Surprise, Sad and Neutral. Overall, 24 observers participated in our collection of the training set, including 19 observers who participated in only one session watching 80 videos from 7 classes and 5 observers who participated multiple times and watched all the videos. All videos were presented in an order balanced fashion. Leave-one-observer-out was employed in this classification task. We report the classification accuracy of our baseline, a three-layer network, on this initial training set while training with signals from all participants, only single participants and only multiple participants. We also investigate the recognition accuracy of grouping the dataset by arousal or valence, which achieves 68.66% and 72.72% separately. Finally, we provide a two-step network. The first step is to classify the features into high/low arousal or positive/negative valence by a network. Then the arousal/valence middle output of the first step is concatenated with feature sets as input of the second step for emotion recognition. We found that adding arousal or valence information can help to improve the classification accuracy. In addition, the information of positive/negative valence boosts the classification accuracy to a higher degree on this dataset.

研究动机与目标

• 开发一个基于真实世界刺激的大规模生理数据集，用于情绪识别。
• 建立基于多位观察者EDA信号的情绪分类基线。
• 探究唤醒度或效价信息是否能提升基于生理信号的情绪识别准确率。
• 评估一种两步神经网络架构的有效性，该架构将唤醒度/效价作为中间预测任务。

提出的方法

• 使用E4腕带和Eye Tribe眼动追踪仪，采集24名观察者观看AFEW数据集中80段电影片段时的生理信号，构建PAFEW数据集。
• 针对每段视频片段的每个EDA信号序列，提取六项特征以表征生理反应。
• 采用三层前馈神经网络作为基线模型，对七种情绪（愤怒、厌恶、恐惧、快乐、惊讶、悲伤和中性）进行分类。
• 采用留一观察者交叉验证策略，确保在不同参与者间评估的稳健性。
• 设计两步网络：第一步对唤醒度（高/低）或效价（正向/负向）进行分类，第二步将中间输出与原始特征拼接后用于最终情绪分类。
• 损失函数同时包含唤醒度/效价分类与情绪分类任务，支持端到端训练并共享超参数。

实验结果

研究问题

• RQ1 当通过电影片段诱发情绪时，生理信号（尤其是EDA）能否可靠地对七种基本情绪进行分类？
• RQ2 将唤醒度或效价作为中间分类任务是否能提升神经网络在EDA情绪识别中的性能？
• RQ3 在该数据集中，效价是否比唤醒度提供更具信息量的辅助信号，从而提升情绪分类准确率？
• RQ4 在单人与多人数据上进行训练时，模型性能如何变化？

主要发现

• 基线三层网络在PAFEW数据集的单名参与者子集上达到42.08%的分类准确率。
• 将情绪分为高唤醒/低唤醒的分类准确率达到68.66%，显著高于随机猜测水平（52.03%）。
• 将情绪分为正向效价/负向效价的分类准确率达到72.72%，比随机猜测水平（50.62%）高出22.10%。
• 当将效价信息作为辅助任务时，两步网络相比基线模型准确率提升了4.9%。
• 添加效价信息带来的性能提升高于添加唤醒度信息，表明在该数据集中，效价是更有效的归纳偏置。
• 单人参与者数据的准确率（42.08%）高于多人参与数据（33.07%），表明重复会话引入了更多噪声。