[Paper Review] Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

Zhaodong Sun, Xiaobai Li|arXiv (Cornell University)|Aug 8, 2022

Non-Invasive Vital Sign Monitoring57 references4 citations

TL;DR

This paper proposes Contrast-Phys, an unsupervised video-based remote photoplethysmography (rPPG) method that leverages spatiotemporal contrastive learning to estimate physiological signals like heart rate from face videos without requiring ground truth labels. By exploiting rPPG signal similarities across spatial and temporal locations within the same video and dissimilarities across different videos, the method achieves performance on par with state-of-the-art supervised rPPG models while being significantly faster and more robust to noise.

ABSTRACT

Video-based remote physiological measurement utilizes face videos to measure the blood volume change signal, which is also called remote photoplethysmography (rPPG). Supervised methods for rPPG measurements achieve state-of-the-art performance. However, supervised rPPG methods require face videos and ground truth physiological signals for model training. In this paper, we propose an unsupervised rPPG measurement method that does not require ground truth signals for training. We use a 3DCNN model to generate multiple rPPG signals from each video in different spatiotemporal locations and train the model with a contrastive loss where rPPG signals from the same video are pulled together while those from different videos are pushed away. We test on five public datasets, including RGB videos and NIR videos. The results show that our method outperforms the previous unsupervised baseline and achieves accuracies very close to the current best supervised rPPG methods on all five datasets. Furthermore, we also demonstrate that our approach can run at a much faster speed and is more robust to noises than the previous unsupervised baseline. Our code is available at https://github.com/zhaodongsun/contrast-phys.

Motivation & Objective

• To develop an unsupervised rPPG method that eliminates the need for expensive ground truth physiological signals during training.
• To improve robustness to periodic noise and head motion, which commonly degrade existing unsupervised rPPG methods.
• To achieve performance comparable to supervised rPPG models using only video data and no labeled physiological signals.
• To enhance training efficiency by avoiding redundant model forward passes used in prior self-supervised approaches.

Proposed method

• The method introduces a spatiotemporal rPPG (ST-rPPG) block representation that extracts multiple rPPG signals across spatial (height, width) and temporal (time) dimensions from a single video.
• It employs a 3D CNN backbone to process video clips and generate ST-rPPG features, enabling joint spatial and temporal modeling of rPPG signals.
• Contrastive learning is applied using a contrastive loss that pulls together rPPG signals from the same video (positive pairs) and pushes apart signals from different videos (negative pairs).
• The method leverages four key rPPG observations: spatial similarity of rPPG signals across facial regions, temporal similarity in short clips, dissimilarity across videos, and the bounded HR range (0.66–4.16 Hz).
• Spatiotemporal sampling is used to generate positive pairs from the same video at different spatial and temporal locations, while negative pairs are formed across different videos.
• The training objective is optimized end-to-end using contrastive loss, with no need for ground truth labels.

Experimental results

Research questions

• RQ1 Can rPPG signals be accurately estimated from face videos without any ground truth physiological labels?
• RQ2 Can spatiotemporal contrastive learning effectively exploit intrinsic signal regularities in rPPG to enable unsupervised representation learning?
• RQ3 How does the proposed method compare in accuracy, speed, and noise robustness to existing unsupervised rPPG baselines?
• RQ4 What is the optimal spatiotemporal resolution and clip length for ST-rPPG block construction?

Key findings

• Contrast-Phys outperforms the previous unsupervised baseline [Gideon2021] on all five public datasets, achieving a mean absolute error (MAE) of 0.64 bpm on UBFC-rPPG, compared to 1.85 bpm for the prior method.
• On the UBFC-rPPG dataset, Contrast-Phys achieves an R-value of 0.995, closely matching the best supervised rPPG methods.
• When periodic noise is injected, Contrast-Phys maintains an MAE of 0.74 bpm and R of 0.991, while Gideon2021 degrades to an MAE of 22.47 bpm and R of 0.244.
• The method runs significantly faster than prior unsupervised approaches due to avoiding redundant forward passes.
• Sensitivity analysis shows that a 2×2 spatial resolution and 10-second temporal length yield optimal performance, balancing receptive field and signal quality.
• Saliency maps confirm that Contrast-Phys correctly focuses on facial skin regions, even under noise or head motion, while Gideon2021 shows erratic and noise-sensitive attention patterns.