[Paper Review] PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography

Rencheng Song, Huan Chen+4 more|arXiv (Cornell University)|Jun 4, 2020

Non-Invasive Vital Sign Monitoring26 references5 citations

TL;DR

PulseGAN proposes a conditional generative adversarial network (cGAN) framework to generate high-fidelity remote photoplethysmography (rPPG) pulse waveforms by denoising CHROM-derived signals using time-domain and spectral-domain losses alongside adversarial training. It significantly improves heart rate variability (HRV) metrics, reducing mean absolute error in AVNN and SDNN by 20.85% and 20.28% respectively compared to DAE in cross-database testing.

ABSTRACT

Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitation of inaccurate pulse signals. In this paper, a new framework based on generative adversarial network, called PulseGAN, is introduced to generate realistic rPPG pulse signals through denoising the chrominance signals. Considering that the cardiac signal is quasi-periodic and has apparent time-frequency characteristics, the error losses defined in time and spectrum domains are both employed with the adversarial loss to enforce the model generating accurate pulse waveforms as its reference. The proposed framework is tested on the public UBFC-RPPG database in both within-database and cross-database configurations. The results show that the PulseGAN framework can effectively improve the waveform quality, thereby enhancing the accuracy of HR, the heart rate variability (HRV) and the interbeat interval (IBI). The proposed method achieves the best performance compared to the denoising autoencoder (DAE) and CHROM, with the mean absolute error of AVNN (the average of all normal-to-normal intervals) improving 20.85% and 41.19%, and the mean absolute error of SDNN (the standard deviation of all NN intervals) improving 20.28% and 37.53%, respectively, in the cross-database test. This framework can be easily extended to other existing deep learning based rPPG methods, which is expected to expand the application scope of rPPG techniques.

Motivation & Objective

• To address the limitation of existing rPPG methods in generating high-quality pulse waveforms necessary for advanced cardiac feature extraction.
• To improve the accuracy of heart rate variability (HRV) metrics such as AVNN and SDNN, which require precise waveform reconstruction.
• To develop a deep learning framework that enhances rPPG signal quality beyond average heart rate estimation.
• To enable integration with existing deep learning-based rPPG methods to boost their performance through waveform refinement.

Proposed method

• A conditional GAN (cGAN) is used, with the generator taking rough CHROM-derived rPPG signals as input and producing refined pulse waveforms.
• The discriminator evaluates generated signals against ground-truth PPG signals from a pulse oximeter, using the input signal as a conditioning factor.
• Adversarial loss is combined with time-domain waveform error loss and frequency-domain spectrum error loss to enforce fidelity in both temporal and spectral characteristics.
• The model is trained end-to-end on the UBFC-RPPG database, with cross-database generalization tested using data from PURE and BSIPL-RPPG.
• The framework is designed to be modular, allowing integration with other deep learning-based rPPG methods for performance enhancement.

Experimental results

Research questions

• RQ1 Can a GAN-based framework generate more realistic rPPG pulse waveforms than conventional denoising or baseline rPPG methods?
• RQ2 Does incorporating time-domain and frequency-domain losses improve waveform fidelity and HRV metric accuracy?
• RQ3 To what extent does the PulseGAN framework generalize across different databases in cross-database testing?
• RQ4 Can the proposed framework be effectively integrated with existing deep learning-based rPPG methods to enhance their output quality?

Key findings

• In cross-database testing, PulseGAN reduced the mean absolute error (MAE) of AVNN by 20.85% compared to DAE and 41.19% compared to CHROM.
• The MAE of SDNN was improved by 20.28% (vs. DAE) and 37.53% (vs. CHROM) in the cross-database scenario.
• The IBI vector MAE was significantly reduced by PulseGAN compared to DAE, with error distribution shifting closer to ground truth.
• Bland-Altman plots confirmed that PulseGAN’s HR predictions showed better consistency with reference values than DAE and CHROM.
• Visual comparisons demonstrated that PulseGAN-generated waveforms and IBI sequences were closer to ground truth than those from DAE and CHROM.
• The model achieved the best performance in HR estimation, with HR MAE of 2.09 bpm and RMSE of 4.42 bpm on the UBFC-RPPG dataset in cross-database evaluation.