[Paper Review] Privacy-Protective-GAN for Face De-identification

Yifan Wu, Fan Yang+1 more|arXiv (Cornell University)|Jun 23, 2018

Face recognition and analysis21 references44 citations

TL;DR

This paper introduces Privacy-Protective-GAN (PP-GAN), a GAN-based framework for face de-identification that removes identity while preserving structure and visual utility using a verificator (contrastive loss) and a regulator (SSIM). It reports strong privacy protection, retained utility, and high visual similarity on MORPH data.

ABSTRACT

Face de-identification has become increasingly important as the image sources are explosively growing and easily accessible. The advance of new face recognition techniques also arises people's concern regarding the privacy leakage. The mainstream pipelines of face de-identification are mostly based on the k-same framework, which bears critiques of low effectiveness and poor visual quality. In this paper, we propose a new framework called Privacy-Protective-GAN (PP-GAN) that adapts GAN with novel verificator and regulator modules specially designed for the face de-identification problem to ensure generating de-identified output with retained structure similarity according to a single input. We evaluate the proposed approach in terms of privacy protection, utility preservation, and structure similarity. Our approach not only outperforms existing face de-identification techniques but also provides a practical framework of adapting GAN with priors of domain knowledge.

Motivation & Objective

• Motivate privacy-preserving face analysis as images proliferate and face recognition grows more capable.
• Develop a GAN-based de-identification framework that balances identity removal with structural/visual utility.
• Incorporate priors of identity (verificator) and structure (SSIM regulator) into GAN training.
• Provide end-to-end trainable architecture that can generate de-identified outputs for individual probe faces.
• Propose systematic evaluation metrics combining privacy (verification) and utility (detection and SSIM).

Proposed method

• Propose PP-GAN with four components: generator (U-Net), discriminator, verificator (contrastive loss), and regulator (SSIM).
• Train a conditional GAN with three loss terms: cGAN loss, verification loss, and SSIM loss.
• Pre-train a verificator (Light CNN-9 with Siamese contrastive loss) and freeze it during PP-GAN training.
• Use SSIM as the structure-preserving loss to encourage luminance, contrast, and structural similarity, while the verificator enforces identity separation.
• Optimize the generator via a combined loss L_face = L_cGAN + lambda1 * L_verif + lambda2 * L_sim, in a min-max (G,D) framework.

Experimental results

Research questions

• RQ1 Can a GAN-based de-identification method effectively remove biometric identity while preserving image structure and utility?
• RQ2 Does integrating a verificator and an SSIM-based regulator improve privacy protection without sacrificing visual quality?
• RQ3 What is the trade-off between identity removal and structural similarity in PP-GAN across diverse demographic groups?
• RQ4 How does PP-GAN perform relative to existing de-identification approaches on standard privacy and utility metrics?

Key findings

• PP-GAN achieves strong privacy protection by hindering verification between original and de-identified images.
• The method preserves data utility, maintaining face-like quality and detector usability after de-identification.
• Incorporating both the verificator and SSIM regulator yields a better balance between identity removal and visual similarity than using either alone.
• Quantitative evaluation on MORPH shows high de-identification performance with preserved structure and utility across gender/race groups.
• The framework introduces a systematic evaluation plan combining a pretrained verifier and face detector to assess de-identification rate and utility.
• The approach outperforms prior k-same and GAN-based methods in terms of de-identification effectiveness and visual quality.