[Paper Review] Py-Feat: Python Facial Expression Analysis Toolbox

Jin Hyun Cheong, Eshin Jolly+4 more|arXiv (Cornell University)|Apr 8, 2021

Emotion and Mood Recognition31 citations

TL;DR

Py-Feat is an open-source Python toolbox that provides end-to-end support for detecting, preprocessing, analyzing, and visualizing facial expression data to help researchers disseminate and benchmark CV models in social science domains.

ABSTRACT

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state of the art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.

Motivation & Objective

• Address the lack of user-friendly, open-source tools for facial expression research in social sciences.
• Provide an end-to-end toolbox to detect, preprocess, analyze, and visualize facial expression data.
• Facilitate dissemination and benchmarking of computer vision models in non-vision-domain research.

Proposed method

• Introduce Py-Feat as an open-source Python toolbox.
• Offer modules for detection, preprocessing, analysis, and visualization of facial expression data.
• Design to be accessible to domain experts and end users for processing and exploring data.

Experimental results

Research questions

• RQ1 How can an open-source toolbox lower the barrier for social science researchers to use facial expression analysis?
• RQ2 Can Py-Feat enable easier dissemination and benchmarking of facial expression models across domains?
• RQ3 What features (detection, preprocessing, analysis, visualization) are most helpful for facial expression research?
• RQ4 Does the toolbox sufficiently integrate components to support end-to-end workflows from raw data to insights?

Key findings

• Py-Feat provides a comprehensive set of tools for detecting, preprocessing, analyzing, and visualizing facial expression data.
• The toolbox aims to ease reuse and benchmarking of computer vision models by domain experts.
• It targets facilitating increased use of facial expression data in human behavior research.
• The platform emphasizes openness and accessibility for researchers in psychology and related fields.