[Paper Review] Faces of Experimental Pain: Transferability of Deep Learned Heat Pain Features to Electrical Pain

Pooja Prajod, Dominik Schiller+2 more|arXiv (Cornell University)|Jun 17, 2024

Neuroscience and Neural Engineering5 citations

TL;DR

The study tests whether a deep heat-pain feature extractor trained on one pain dataset can transfer to electrical pain recognition, using AI4Pain facial videos to train simple ANN and LSTM classifiers that outperform baselines.

ABSTRACT

The limited size of pain datasets are a challenge in developing robust deep learning models for pain recognition. Transfer learning approaches are often employed in these scenarios. In this study, we investigate whether deep learned feature representation for one type of experimentally induced pain can be transferred to another. Participating in the AI4Pain challenge, our goal is to classify three levels of pain (No-Pain, Low-Pain, High-Pain). The challenge dataset contains data collected from 65 participants undergoing varying intensities of electrical pain. We utilize the video recording from the dataset to investigate the transferability of deep learned heat pain model to electrical pain. In our proposed approach, we leverage an existing heat pain convolutional neural network (CNN) - trained on BioVid dataset - as a feature extractor. The images from the challenge dataset are inputted to the pre-trained heat pain CNN to obtain feature vectors. These feature vectors are used to train two machine learning models: a simple feed-forward neural network and a long short-term memory (LSTM) network. Our approach was tested using the dataset's predefined training, validation, and testing splits. Our models outperformed the baseline of the challenge on both the validation and tests sets, highlighting the potential of models trained on other pain datasets for reliable feature extraction.

Motivation & Objective

• Address the data scarcity problem in pain recognition by leveraging transfer learning from a heat-pain model to an electrical-pain dataset.
• Evaluate whether facial features learned from heat pain can effectively classify three levels of electrical-pain (No-Pain, Low-Pain, High-Pain) in the AI4Pain dataset.
• Assess performance of two lightweight target models (ANN with voting and LSTM) using features from a pre-trained heat-pain CNN.
• Compare transfer-based models against the AI4Pain challenge baselines on validation and test splits.

Proposed method

• Use a pre-trained VGG16-based CNN (modified with Global Average Pooling) trained on BioVid heat-pain data as a 1024-dim feature extractor for face crops from AI4Pain videos.
• Crop, align, and resize faces to 224x224; sample videos at 30 Hz and extract frame-level feature vectors.
• Train two target models on the extracted features: (i) a simple four-layer ANN with majority voting for video-level prediction; (ii) a two-layer LSTM network processing 300-frame sequences (10 seconds at 30 Hz) with batch normalization and dropout.
• Evaluate on the AI4Pain training/validation/test splits and report accuracy, precision, recall, and F1-score; compare against baselines (Video, fNIRS, and Video+fNIRS).

Experimental results

Research questions

• RQ1 Can features learned from heat pain facial expressions generalize to electrical pain for three-class classification (No-, Low-, High-Pain)?
• RQ2 Do transfer-based models (ANN and LSTM) outperform the AI4Pain baselines on validation and test sets when using cross-domain pain features?
• RQ3 Is frame-level CNN-derived feature transfer robust to cross-stimulus differences (heat vs electrical pain) in facial expressions?
• RQ4 What is the impact of using time-series modelling (LSTM) versus frame-wise ANN with majority voting in this transfer setting?

Key findings

• Both transfer-based models outperformed the challenge baselines on the validation set; LSTM achieved the highest validation accuracy of 0.60.
• On the test set, the simple ANN with voting outperformed all baselines (0.49) and the LSTM (0.43).
• The unimodal fNIRS baseline performed best among baselines on validation, while the multimodal combination did not improve over unimodal baselines.
• The results suggest that heat-pain learned features can be repurposed to recognize electrical-pain expressions in facial videos, aiding pain recognition with limited data.