[Paper Review] A Study of Face Obfuscation in ImageNet

Kaiyu Yang, Jacqueline Yau+3 more|arXiv (Cornell University)|Mar 10, 2021

Face recognition and analysis78 references66 citations

TL;DR

The paper annotates faces in ImageNet and shows that face obfuscation (blurring or overlaying) has only a small impact on classification accuracy and transfer learning performance. It establishes a privacy-enhanced ImageNet variant and analyzes category- and feature-transfer effects.

ABSTRACT

Face obfuscation (blurring, mosaicing, etc.) has been shown to be effective for privacy protection; nevertheless, object recognition research typically assumes access to complete, unobfuscated images. In this paper, we explore the effects of face obfuscation on the popular ImageNet challenge visual recognition benchmark. Most categories in the ImageNet challenge are not people categories; however, many incidental people appear in the images, and their privacy is a concern. We first annotate faces in the dataset. Then we demonstrate that face obfuscation has minimal impact on the accuracy of recognition models. Concretely, we benchmark multiple deep neural networks on obfuscated images and observe that the overall recognition accuracy drops only slightly (<= 1.0%). Further, we experiment with transfer learning to 4 downstream tasks (object recognition, scene recognition, face attribute classification, and object detection) and show that features learned on obfuscated images are equally transferable. Our work demonstrates the feasibility of privacy-aware visual recognition, improves the highly-used ImageNet challenge benchmark, and suggests an important path for future visual datasets. Data and code are available at https://github.com/princetonvisualai/imagenet-face-obfuscation.

Motivation & Objective

• Motivate privacy concerns for public visual datasets by studying ubiquitous faces in ImageNet (ILSVRC).
• Annotate faces in ILSVRC to enable obfuscation and privacy-preserving benchmarks.
• Quantify the impact of face obfuscation on ImageNet classification and transfer learning.
• Assess whether features learned on obfuscated data transfer to downstream tasks.
• Provide a dataset and methodology to facilitate privacy-aware visual recognition research.

Proposed method

• Develop a two-stage face annotation pipeline combining automatic detection (Amazon Rekognition) with crowdsourced refinement (Mechanical Turk).
• Create face-obfuscated versions of ILSVRC using blurring and overlaying techniques.
• Benchmark multiple deep networks on original and obfuscated images across Top-1 and Top-5 accuracy.
• Evaluate transfer learning by pretraining on original/obfuscated images and finetuning on four downstream tasks (CIFAR-10, SUN, PASCAL VOC, CelebA).
• Analyze category-wise robustness and use Grad-CAM for localization insight.
• Provide data and code for privacy-aware ImageNet research.

Experimental results

Research questions

• RQ1 How does face obfuscation affect ImageNet classification accuracy across common CNN architectures?
• RQ2 Does pretraining on face-obfuscated ImageNet preserve transfer performance to object, scene, detection, and face-attribute tasks?
• RQ3 Which categories are most sensitive to face obfuscation, and how does object-face overlap influence degradation?
• RQ4 Can privacy-preserving obfuscation be integrated into large-scale visual benchmarks without significantly harming utility?

Key findings

• Face obfuscation causes only a small accuracy drop: top-1 and top-5 declines are typically within 0.1%–1.0% depending on method and model.
• Across 1000 ImageNet categories, the overall mean accuracy drop from obfuscation is less than 1% for both blurring and overlaying.
• Pretraining on face-obfuscated images yields transfer performance comparable to pretraining on original images across object recognition (CIFAR-10), scene recognition (SUN), object detection (PASCAL VOC), and face attribute classification (CelebA).
• Categories overlapping with faces (e.g., mask, harmonica) show larger degradation under obfuscation, with correlation between blurred-face fraction and accuracy drop (top-5 r≈0.46, p≈2.69e-49).
• Grad-CAM analyses reveal that obfuscated images reduce model focus on target objects, explaining some category-specific drops.
• Disparate changes between visually similar categories (e.g., eskimo dog vs. siberian husky) under obfuscation are largely mitigated when evaluating top-5 accuracy or average precision.