[Paper Review] Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation

Bingquan Zhu, Hao Fang+2 more|arXiv (Cornell University)|Feb 7, 2020

Face recognition and analysis24 references4 citations

TL;DR

This paper proposes using face-swapping deepfake technology for medical video de-identification to protect patient privacy while preserving diagnostic information. By replacing patients' faces with anonymized target faces, the method maintains near-identical body and facial keypoint locations—significantly outperforming traditional blurring or masking methods—enabling ethical, high-fidelity data sharing for movement disorder research.

ABSTRACT

Data sharing for medical research has been difficult as open-sourcing clinical data may violate patient privacy. Traditional methods for face de-identification wipe out facial information entirely, making it impossible to analyze facial behavior. Recent advancements on whole-body keypoints detection also rely on facial input to estimate body keypoints. Both facial and body keypoints are critical in some medical diagnoses, and keypoints invariability after de-identification is of great importance. Here, we propose a solution using deepfake technology, the face swapping technique. While this swapping method has been criticized for invading privacy and portraiture right, it could conversely protect privacy in medical video: patients' faces could be swapped to a proper target face and become unrecognizable. However, it remained an open question that to what extent the swapping de-identification method could affect the automatic detection of body keypoints. In this study, we apply deepfake technology to Parkinson's disease examination videos to de-identify subjects, and quantitatively show that: face-swapping as a de-identification approach is reliable, and it keeps the keypoints almost invariant, significantly better than traditional methods. This study proposes a pipeline for video de-identification and keypoint preservation, clearing up some ethical restrictions for medical data sharing. This work could make open-source high quality medical video datasets more feasible and promote future medical research that benefits our society.

Motivation & Objective

• To address the ethical and privacy barriers in sharing medical videos containing identifiable patient faces.
• To preserve critical diagnostic information, such as body and facial keypoints, during de-identification.
• To evaluate whether deepfake-based face swapping maintains keypoint invariance compared to traditional de-identification methods.
• To ensure the de-identified videos are irreversible to prevent re-identification of original subjects.
• To establish a feasible pipeline for open-sourcing high-quality medical video datasets while protecting patient identity.

Proposed method

• Face-swapping via a deep neural network is applied to Parkinson’s disease examination videos to replace original patient faces with anonymized, open-source faces.
• The method uses a pre-trained deepfake model (e.g., Faceswap) to transfer facial features while preserving expressions, skin tone, and head pose.
• Keypoint detection is performed using OpenPose, a bottom-up pose estimation model with multi-scale receptive fields, to assess stability across methods.
• The performance of keypoint detection is quantitatively compared between original, blurred/masked, and face-swapped videos.
• Irreversibility is evaluated by analyzing whether original faces can be reconstructed from public de-identified videos without access to model weights or original data.
• The pipeline is tested across diverse recording conditions, including varying lighting, poses, and facial expressions.

Experimental results

Research questions

• RQ1 To what extent does face-swapping deepfake technology preserve body and facial keypoint locations compared to traditional de-identification methods?
• RQ2 Can deepfake-based de-identification maintain diagnostic information integrity while ensuring patient privacy?
• RQ3 Is the face-swapped video data irreversible, preventing reconstruction of original identities?
• RQ4 How robust is the keypoint preservation across different subjects and video recording conditions?
• RQ5 Can this method enable ethical, open-sharing of high-value medical video datasets without compromising data utility?

Key findings

• Face-swapping de-identification preserves facial and body keypoints with near-identical accuracy to original videos, significantly outperforming blurring and masking methods.
• Traditional methods like blurring or masking caused near-total failure in facial keypoint detection due to complete loss of facial data.
• Body keypoint detection remained stable across all methods, but face-swapping showed the highest invariance, especially for facial landmarks.
• The deepfake model’s weights are not publicly available, making reconstruction of original faces from swapped videos computationally infeasible.
• The method is robust across diverse facial expressions, lighting, and poses, maintaining keypoint consistency in real-world medical video conditions.
• This approach enables ethical data sharing of high-quality medical videos without compromising diagnostic information or privacy.