[Paper Review] Exploring Facial Expressions and Affective Domains for Parkinson Detection

Luis F. Gomez, Aythami Morales+2 more|arXiv (Cornell University)|Dec 11, 2020

Facial Nerve Paralysis Treatment and Research47 references4 citations

TL;DR

This paper proposes a deep learning framework that leverages affective domain adaptation and triplet loss to improve Parkinson's disease (PD) detection by analyzing facial expressions from video sequences. By modeling dynamic facial changes across emotion phases (neutral, onset-transition, apex, offset-transition, neutral), the method achieves 87.3% accuracy—up to 8.9% higher than single-image baselines—demonstrating that temporal dynamics and transfer learning from affective models significantly enhance hypomimia detection.

ABSTRACT

Parkinson's Disease (PD) is a neurological disorder that affects facial movements and non-verbal communication. Patients with PD present a reduction in facial movements called hypomimia which is evaluated in item 3.2 of the MDS-UPDRS-III scale. In this work, we propose to use facial expression analysis from face images based on affective domains to improve PD detection. We propose different domain adaptation techniques to exploit the latest advances in face recognition and Face Action Unit (FAU) detection. The principal contributions of this work are: (1) a novel framework to exploit deep face architectures to model hypomimia in PD patients; (2) we experimentally compare PD detection based on single images vs. image sequences while the patients are evoked various face expressions; (3) we explore different domain adaptation techniques to exploit existing models initially trained either for Face Recognition or to detect FAUs for the automatic discrimination between PD patients and healthy subjects; and (4) a new approach to use triplet-loss learning to improve hypomimia modeling and PD detection. The results on real face images from PD patients show that we are able to properly model evoked emotions using image sequences (neutral, onset-transition, apex, offset-transition, and neutral) with accuracy improvements up to 5.5% (from 72.9% to 78.4%) with respect to single-image PD detection. We also show that our proposed affective-domain adaptation provides improvements in PD detection up to 8.9% (from 78.4% to 87.3% detection accuracy).

Motivation & Objective

• To develop an automated, objective system for Parkinson’s disease detection using facial expression dynamics.
• To investigate whether modeling temporal phases of facial emotion expression improves detection over single-image analysis.
• To explore domain adaptation from pre-trained face recognition and FAU detection models to enhance PD detection performance.
• To evaluate the effectiveness of triplet loss in improving feature representation for hypomimia modeling.
• To assess the feasibility of lightweight models for deployment on portable devices.

Proposed method

• The framework processes video sequences segmented into five emotion phases: neutral, onset-transition, apex, offset-transition, and neutral, to capture dynamic facial changes.
• Affective domain adaptation is applied by fine-tuning pre-trained deep face models (e.g., Freeze 75) using transfer learning on PD and healthy subject data.
• Triplet loss is employed to optimize feature embeddings, encouraging closer clustering of similar samples (e.g., PD patients) and larger margins between different classes.
• The method compares single-frame classification against multi-phase frame analysis to evaluate the impact of temporal dynamics.
• Reduced architectures (VGG-8, ResNet7) are evaluated for portability, with triplet loss further improving performance on these models.
• Principal component analysis is used to visualize the separability of PD patients across different impairment levels in the learned feature space.

Experimental results

Research questions

• RQ1 Does modeling facial expression dynamics across multiple phases of emotion production improve PD detection accuracy compared to single-image analysis?
• RQ2 Can transfer learning from pre-trained models in face recognition or FAU detection domains enhance PD detection performance?
• RQ3 To what extent does triplet loss learning improve feature representation for hypomimia modeling in PD patients?
• RQ4 Can lightweight deep learning models trained with triplet loss achieve acceptable accuracy for deployment on portable devices?
• RQ5 Is there a discernible trend in the feature space that reflects the neurological severity of PD patients?

Key findings

• Using video sequences with multiple emotion phases improved PD detection accuracy by 5.5 percentage points, rising from 72.9% to 78.4% compared to single-image methods.
• Affective domain adaptation via transfer learning boosted detection accuracy by 8.9 percentage points, reaching 87.3% when using models fine-tuned on emotion data.
• The VGG-8 and ResNet7 models achieved maximum accuracies of 67.6% and 78.8% respectively, which were further improved to 72.7% and 82.4% using triplet loss.
• The principal component space revealed a clear trend in feature separability, with mild and severe PD patients forming distinct clusters and intermediate cases lying between them.
• Triplet loss learning significantly enhanced feature representation, particularly in lightweight models, suggesting its value for efficient and accurate PD detection.
• The tri-class classification of PD patients into mild, moderate, and severe groups achieved a maximum accuracy of 44% (F1=0.45), indicating room for improvement in severity estimation.