[Paper Review] IdentityDP: Differential Private Identification Protection for Face Images

Yunqian Wen, Song Li+3 more|arXiv (Cornell University)|Mar 2, 2021

Face recognition and analysis4 citations

TL;DR

This paper proposes IdentityDP, a novel face de-identification framework that integrates differential privacy (DP) with a disentangled GAN to anonymize face images while preserving visual quality and enabling adjustable privacy-utility trade-offs. By applying DP perturbations to identity-specific features and reconstructing images via a fixed generator, IdentityDP achieves high perceptual similarity and strong privacy guarantees without requiring pre-annotations or external identity guidance.

ABSTRACT

Because of the explosive growth of face photos as well as their widespread dissemination and easy accessibility in social media, the security and privacy of personal identity information becomes an unprecedented challenge. Meanwhile, the convenience brought by advanced identity-agnostic computer vision technologies is attractive. Therefore, it is important to use face images while taking careful consideration in protecting people's identities. Given a face image, face de-identification, also known as face anonymization, refers to generating another image with similar appearance and the same background, while the real identity is hidden. Although extensive efforts have been made, existing face de-identification techniques are either insufficient in photo-reality or incapable of well-balancing privacy and utility. In this paper, we focus on tackling these challenges to improve face de-identification. We propose IdentityDP, a face anonymization framework that combines a data-driven deep neural network with a differential privacy (DP) mechanism. This framework encompasses three stages: facial representations disentanglement, $ε$-IdentityDP perturbation and image reconstruction. Our model can effectively obfuscate the identity-related information of faces, preserve significant visual similarity, and generate high-quality images that can be used for identity-agnostic computer vision tasks, such as detection, tracking, etc. Different from the previous methods, we can adjust the balance of privacy and utility through the privacy budget according to pratical demands and provide a diversity of results without pre-annotations. Extensive experiments demonstrate the effectiveness and generalization ability of our proposed anonymization framework.

Motivation & Objective

• Address the growing privacy risks posed by widespread face image sharing in social media and public datasets.
• Overcome limitations of existing face anonymization methods, including poor visual quality, reliance on target system access, and dependence on pre-annotations.
• Enable adjustable privacy-utility trade-offs through a differentiable, data-driven framework that preserves identity-agnostic utility for computer vision tasks.
• Provide a generalizable, annotation-free solution that maintains visual fidelity across diverse face images, including challenging lighting conditions and artistic portraits.

Proposed method

• Employ a disentangled GAN to separate identity representations from multi-level attribute representations in the latent feature space.
• Apply ε-IdentityDP perturbations directly to the disentangled identity representation to ensure formal differential privacy guarantees.
• Fix the trained GAN generator and reconstruct anonymized images using the perturbed identity code and unchanged attribute codes.
• Use a noise tensor with variance calibrated to the privacy budget ε to control the level of obfuscation.
• Train the disentanglement GAN using adversarial loss and identity reconstruction loss to maintain visual similarity.
• Integrate the DP mechanism at inference time, allowing users to adjust ε for desired privacy-utility balance.

Experimental results

Research questions

• RQ1 Can a deep generative model combined with differential privacy effectively anonymize face images while preserving high visual fidelity?
• RQ2 How does the proposed IdentityDP framework balance privacy protection and image utility compared to state-of-the-art methods?
• RQ3 To what extent can IdentityDP generalize to unseen face images, including those under challenging lighting or artistic styles?
• RQ4 Can IdentityDP achieve privacy protection without relying on pre-annotations, external identity providers, or access to target system parameters?

Key findings

• IdentityDP achieves superior visual similarity compared to CIAGAN, as shown in qualitative comparisons on CelebA and NIST challenge images.
• Quantitative results show that IdentityDP maintains higher image utility (measured by FID and LPIPS) than CIAGAN while offering stronger privacy protection.
• Fawkes, an adversarial perturbation-based method, performs poorly on privacy metrics despite high visual similarity, indicating weak privacy guarantees.
• IdentityDP generalizes well to challenging inputs, including low-light, high-contrast, and artistic portrait images, with no visible artifacts or distortions.
• The framework incurs minimal computational overhead, processing images in an average of 0.329 seconds per image on an NVIDIA GTX 1080 Ti.
• The method supports adjustable privacy budgets (ε), enabling diverse anonymization outcomes without retraining or pre-annotations.