[Paper Review] Automated facial recognition system using deep learning for pain assessment in adults with cerebral palsy

Álvaro Sabater-Gárriz, F. Xavier Gaya-Morey+4 more|arXiv (Cornell University)|Jan 22, 2024

Cerebral Palsy and Movement DisordersMedicine3 citations

TL;DR

This study proposes a deep learning-based automated facial recognition system for pain assessment in adults with cerebral palsy, using a newly curated dataset (CPPAIN) of 109 facial images labeled via the Facial Action Coding System. InceptionV3 achieved 62.67% accuracy and 61.12% F1 score on the CPPAIN dataset, demonstrating feasibility for detecting pain in populations with communication and facial expression impairments.

ABSTRACT

Background: Pain assessment in individuals with neurological conditions, especially those with limited self-report ability and altered facial expressions, presents challenges. Existing measures, relying on direct observation by caregivers, lack sensitivity and specificity. In cerebral palsy, pain is a common comorbidity and a reliable evaluation protocol is crucial. Thus, having an automatic system that recognizes facial expressions could be of enormous help when diagnosing pain in this type of patient. Objectives: 1) to build a dataset of facial pain expressions in individuals with cerebral palsy, and 2) to develop an automated facial recognition system based on deep learning for pain assessment addressed to this population. Methods: Ten neural networks were trained on three pain image databases, including the UNBC-McMaster Shoulder Pain Expression Archive Database, the Multimodal Intensity Pain Dataset, and the Delaware Pain Database. Additionally, a curated dataset (CPPAIN) was created, consisting of 109 preprocessed facial pain expression images from individuals with cerebral palsy, categorized by two physiotherapists using the Facial Action Coding System observational scale. Results: InceptionV3 exhibited promising performance on the CP-PAIN dataset, achieving an accuracy of 62.67% and an F1 score of 61.12%. Explainable artificial intelligence techniques revealed consistent essential features for pain identification across models. Conclusion: This study demonstrates the potential of deep learning models for robust pain detection in populations with neurological conditions and communication disabilities. The creation of a larger dataset specific to cerebral palsy would further enhance model accuracy, offering a valuable tool for discerning subtle and idiosyncratic pain expressions. The insights gained could extend to other complex neurological conditions.

Motivation & Objective

• To address the challenge of reliable pain assessment in adults with cerebral palsy who often cannot self-report due to communication and motor impairments.
• To overcome the limitations of current observational pain scales, which lack sensitivity and specificity in clinical settings.
• To create a dedicated dataset of facial pain expressions in individuals with cerebral palsy for training and evaluating automated systems.
• To develop and evaluate a deep learning-based facial recognition model tailored for detecting pain in this neurologically impaired population.
• To explore explainable AI techniques to identify consistent facial features associated with pain across models.

Proposed method

• Collected and curated a new dataset (CPPAIN) consisting of 109 preprocessed facial images from adults with cerebral palsy.
• Labeled facial expressions in CPPAIN using the Facial Action Coding System (FACS) by two trained physiotherapists to ensure reliability.
• Trained ten deep neural networks on three public pain databases: UNBC-McMaster, Multimodal Intensity Pain Dataset, and Delaware Pain Database.
• Fine-tuned the InceptionV3 architecture on the CPPAIN dataset for pain classification, using transfer learning to adapt to the target population.
• Applied explainable AI techniques (e.g., Grad-CAM) to interpret model predictions and identify salient facial regions for pain detection.
• Evaluated model performance using standard metrics: accuracy, F1 score, and class-wise precision/recall on the CPPAIN test set.

Experimental results

Research questions

• RQ1 Can a deep learning model trained on a new, CP-specific facial pain dataset achieve reliable pain classification in adults with cerebral palsy?
• RQ2 Which deep learning architecture performs best on the CPPAIN dataset for pain expression recognition?
• RQ3 What facial features are consistently identified by explainable AI methods as indicative of pain across different models?
• RQ4 How does the performance of models trained on general pain datasets transfer to the specific context of cerebral palsy?
• RQ5 To what extent can automated systems improve the sensitivity and specificity of pain assessment compared to direct observational methods?

Key findings

• InceptionV3 achieved the highest performance on the CPPAIN dataset with 62.67% accuracy and 61.12% F1 score, outperforming other models tested.
• Explainable AI techniques revealed consistent activation patterns in the eye, brow, and mouth regions, indicating these areas as key for pain detection.
• The model demonstrated robustness in identifying subtle and idiosyncratic pain expressions common in individuals with cerebral palsy.
• The study confirms the feasibility of using deep learning for automated pain assessment in populations with neurological conditions and communication disabilities.
• The creation of a dedicated, high-quality dataset like CPPAIN is essential for improving model accuracy and generalization in clinical applications.
• Future improvements are expected with larger, more diverse datasets specific to cerebral palsy, enabling better detection of nuanced pain expressions.