[Paper Review] A Review on Facial Micro-Expressions Analysis: Datasets, Features and Metrics

Walied Merghani, Adrian K. Davison+1 more|arXiv (Cornell University)|May 7, 2018

Emotion and Mood Recognition73 references48 citations

TL;DR

A comprehensive survey of facial micro-expressions datasets, feature representations, and evaluation metrics, highlighting challenges and future directions for automatic micro-expression analysis.

ABSTRACT

Facial micro-expressions are very brief, spontaneous facial expressions that appear on the face of humans when they either deliberately or unconsciously conceal an emotion. Micro-expression has shorter duration than macro-expression, which makes it more challenging for human and machine. Over the past ten years, automatic micro-expressions recognition has attracted increasing attention from researchers in psychology, computer science, security, neuroscience and other related disciplines. The aim of this paper is to provide the insights of automatic micro-expressions and recommendations for future research. There has been a lot of datasets released over the last decade that facilitated the rapid growth in this field. However, comparison across different datasets is difficult due to the inconsistency in experiment protocol, features used and evaluation methods. To address these issues, we review the datasets, features and the performance metrics deployed in the literature. Relevant challenges such as the spatial temporal settings during data collection, emotional classes versus objective classes in data labelling, face regions in data analysis, standardisation of metrics and the requirements for real-world implementation are discussed. We conclude by proposing some promising future directions to advancing micro-expressions research.

Motivation & Objective

• Summarize publicly available facial micro-expression datasets and categorize them by spontaneity, sampling rate, and labeling.
• Review feature representations used for micro-expression recognition and their historical progression.
• Analyze performance metrics and evaluation protocols used in micro-expression literature.
• Identify key challenges and standardization gaps affecting cross-study comparability.
• Provide recommendations and future directions for advancing micro-expression research.

Proposed method

• Catalog datasets (spontaneous and non-spontaneous) and compare key properties (fps, resolution, participants, emotions, FACS coding).
• Survey feature extraction methods (3D HOG, LBP-TOP, HOOF, GDs, deep learning) and their evolution over time.
• Summarize performance metrics and evaluation setups (LOS0, LOVO, etc.) reported across studies.
• Discuss data collection challenges (stimulus design, high-speed capture, ethical aspects) and labeling issues (emotion vs. objective classes).
• Synthesize findings to propose directions for datasets, standards, and real-world applicability.

Experimental results

Research questions

• RQ1 What publicly available micro-expression datasets exist and how do they differ in spontaneity, frame rate, and labeling?
• RQ2 Which feature representations have been most effective for micro-expression recognition, and how has this evolved?
• RQ3 What evaluation protocols and metrics are used, and how do they affect cross-study comparability?
• RQ4 What are the main challenges hindering real-world deployment of micro-expression analysis, and how can future work address them?
• RQ5 What recommendations can be made to standardize datasets, features, and metrics for future research?

Key findings

• Non-spontaneous datasets are less suitable for real-world applications due to posing and limited public availability.
• Spontaneous datasets like CASME II and SAMM offer high frame rates and diverse participants, making them favorable for recognition tasks.
• LBP-TOP has been a dominant feature in early micro-expression research, with 3DHOG and HOOF contributing in earlier and later works respectively; deep learning is emerging but still limited by data size.
• Performance across datasets varies significantly due to protocol differences, emphasizing the need for standardized benchmarks and metrics.
• Diversity in participants, high temporal/spatial resolution, and FACS labeling are key advantages of CASME II and SAMM for micro-expression research.