[论文解读] Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

Shervin Minaee, AmirAli Abdolrashidi|arXiv (Cornell University)|Feb 3, 2019

Emotion and Mood Recognition被引用 85

一句话总结

呈现一个带有空间变换器的注意力CNN，聚焦于显著的面部区域，在FER2013、CK+、JAFFE 和 FERG 数据集上实现强劲的 FER 结果。

ABSTRACT

Facial expression recognition has been an active research area over the past few decades, and it is still challenging due to the high intra-class variation. Traditional approaches for this problem rely on hand-crafted features such as SIFT, HOG and LBP, followed by a classifier trained on a database of images or videos. Most of these works perform reasonably well on datasets of images captured in a controlled condition, but fail to perform as good on more challenging datasets with more image variation and partial faces. In recent years, several works proposed an end-to-end framework for facial expression recognition, using deep learning models. Despite the better performance of these works, there still seems to be a great room for improvement. In this work, we propose a deep learning approach based on attentional convolutional network, which is able to focus on important parts of the face, and achieves significant improvement over previous models on multiple datasets, including FER-2013, CK+, FERG, and JAFFE. We also use a visualization technique which is able to find important face regions for detecting different emotions, based on the classifier's output. Through experimental results, we show that different emotions seems to be sensitive to different parts of the face.

研究动机与目标

• 解决面部表情识别（FER）中的高同类内变异。
• 利用注意力聚焦于对情感检测重要的显著面部区域。
• 证明一个浅层、从头训练的网络在 FER 任务上可以超越更深的模型。
• 提供对显著面部区域的可视化以解释模型决策。

提出的方法

• 端到端的注意力卷积网络，包含四层卷积、最大池化、ReLU、 dropout，以及用于特征提取和分类的两层全连接层。
• 将空间变换网络作为定位模块，以估计仿射变换 T(θ) 并对输入进行扭曲以关注重要区域。
• 通过随机优化（Adam）进行训练，损失函数为 L_overall = L_classifier + λ||w(fc)||^2，并结合数据增强。
• 为每个数据集分别训练模型，以应对数据集特定的变化和有限的数据量。

实验结果

研究问题

• RQ1 注意力 CNN 是否在标准 FER 基准上超越更深层网络？
• RQ2 根据模型，哪些面部区域在检测不同情感时最为显著？
• RQ3 所提方法在 FER2013、CK+、JAFFE、FERG 数据集上的表现如何？
• RQ4 在小型 FER 数据集上从头训练并引入注意力的影响是什么？

主要发现

• FER2013 测试精度约为 70.02%。
• JAFFE 数据集精度约为 92.8%。
• CK+ 数据集精度约为 98.0%。
• FERG 数据集精度约为 99.3%。
• 可视化显示情感特异性的显著区域（如开心时的口部；愤怒时的眼睛/眉毛），并指出图像之间的变异性。