[Paper Review] MERANet: Facial Micro-Expression Recognition using 3D Residual Attention Network.

G.V. Subba Reddy, Sai Reddy+2 more|arXiv (Cornell University)|Dec 7, 2020

Emotion and Mood Recognition42 references4 citations

TL;DR

MERANet is a 3D residual attention network for facial micro-expression recognition that integrates spatial-temporal and channel attention mechanisms to capture fine-grained spatio-temporal features. By leveraging 3D kernels, residual connections, and dual attention recalibration, it achieves state-of-the-art performance on benchmark micro-expression datasets.

ABSTRACT

We propose a facial micro-expression recognition model using 3D residual attention network called MERANet. The proposed model takes advantage of spatial-temporal attention and channel attention together, to learn deeper fine-grained subtle features for classification of emotions. The proposed model also encompasses both spatial and temporal information simultaneously using the 3D kernels and residual connections. Moreover, the channel features and spatio-temporal features are re-calibrated using the channel and spatio-temporal attentions, respectively in each residual module. The experiments are conducted on benchmark facial micro-expression datasets. A superior performance is observed as compared to the state-of-the-art for facial micro-expression recognition.

Motivation & Objective

• To address the challenge of recognizing subtle, fleeting facial micro-expressions that are difficult to detect due to minimal visual changes.
• To improve feature representation by jointly modeling spatial, temporal, and channel-wise dependencies in micro-expression sequences.
• To enhance classification performance through attention mechanisms that recalibrate features at both spatial-temporal and channel levels.
• To develop a deep learning architecture that effectively captures fine-grained dynamics in facial micro-expressions using 3D convolutions and residual learning.

Proposed method

• The model employs 3D convolutional kernels to simultaneously extract spatial and temporal features from video sequences of micro-expressions.
• Residual connections are used to stabilize training and enable deeper network architectures for better feature learning.
• Spatial-temporal attention is applied within each residual block to recalibrate spatio-temporal features based on their importance.
• Channel attention is integrated to recalibrate channel-wise feature maps, emphasizing discriminative channels for classification.
• The dual attention mechanism (spatial-temporal and channel) is applied in every residual module to refine features progressively.
• The network is trained end-to-end on benchmark micro-expression datasets using standard classification loss functions.

Experimental results

Research questions

• RQ1 Can a 3D residual network with dual attention mechanisms improve the recognition of subtle facial micro-expressions?
• RQ2 How effectively can spatial-temporal and channel attention mechanisms enhance feature representation in micro-expression recognition?
• RQ3 Does the integration of 3D convolutions and residual learning lead to better performance on benchmark micro-expression datasets?
• RQ4 To what extent does attention-based feature recalibration reduce noise and emphasize discriminative facial motion patterns?

Key findings

• MERANet achieves superior performance compared to state-of-the-art methods on benchmark facial micro-expression datasets.
• The integration of spatial-temporal and channel attention mechanisms significantly improves feature discrimination for micro-expression classification.
• 3D convolutions effectively capture both spatial and temporal dynamics in micro-expression sequences, enhancing model sensitivity.
• Residual connections contribute to stable training and deeper feature learning, leading to improved accuracy.
• The dual attention mechanism enables the model to focus on relevant facial regions and motion patterns, reducing the impact of irrelevant variations.