[论文解读] Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition

Zhengyao Wen, Wenzhong Lin+2 more|arXiv (Cornell University)|Sep 15, 2021

Emotion and Mood Recognition参考文献 52被引用 42

一句话总结

Introduces the Distract Your Attention Network (DAN) for facial expression recognition, combining a Feature Clustering Network, a Multi-head Attention Network, and an Attention Fusion Network to capture diverse local facial regions and maximize class separability.

ABSTRACT

We present a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations. Firstly, multiple classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions exhibit themselves through multiple facial regions simultaneously, and the recognition requires a holistic approach by encoding high-order interactions among local features. To address these issues, we propose our DAN with three key components: Feature Clustering Network (FCN), Multi-head cross Attention Network (MAN), and Attention Fusion Network (AFN). The FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, the MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, the AFN distracts these attentions to multiple locations before fusing the attention maps to a comprehensive one. Extensive experiments on three public datasets (including AffectNet, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. Code will be made available at https://github.com/yaoing/DAN.

研究动机与目标

• 通过识别相似面部表情中的微妙差异来推进 FER。
• 提出一种判别性特征聚类方法，以最大化类间边界并最小化类内方差。
• 开发一种多头注意力机制，同时关注多个面部区域。
• 引入注意力融合机制，在各区域分配注意力并融合特征用于分类。

提出的方法

• Feature Clustering Network (FCN) 通过亲和损失拉近特征到类中心，同时扩大中心之间的边距，从而最大化类可分性。
• Multi-head Attention Network (MAN) 具有若干并行注意力头，每个头包含一个空间注意力单元（多尺度卷积）和一个通道注意力单元，以捕捉高阶交互。
• Attention Fusion Network (AFN) 通过对数-softmax 缩放注意力向量，使用分区损失强制非重叠的注意力区域，并融合注意力特征以用于最终分类。
• 总体训练目标将 affinity loss、partition loss 和 cross-entropy loss 结合为 L = lambda1 L_af + lambda2 L_pt + L_cls。

实验结果

研究问题

• RQ1 判别性特征聚类目标是否能在 FER 中提升超越 center loss 的类间分离？
• RQ2 捕捉多个面部区域的多头注意力结构是否比单头注意力提高 FER 精度？
• RQ3 注意力融合机制是否能促进多样且非重叠的注意力并提升最终分类？
• RQ4 所提出的损失对标准 FER 基准的性能有何影响？

主要发现

• DAN 在 AffectNet-8 上达到 62.09%，在 AffectNet-7 上达到 65.69%，超过若干方法。
• DAN 在 RAF-DB 上达到 89.70%，为该数据集的最新研究成果。
• DAN 在 SFEW 2.0 上达到 53.18%，在较小数据集上显示出竞争力。
• 消融实验表明 affinity loss 与 partition loss 各自对性能提升有贡献。
• 与更少的头相比，四个注意力头在 RAF-DB 上达到最高准确率。