[Paper Review] Fawkes: Protecting Privacy against Unauthorized Deep Learning Models

Shawn Shan, Emily Wenger+4 more|arXiv (Cornell University)|Feb 19, 2020

Adversarial Robustness in Machine LearningComputer Science53 references49 citations

TL;DR

Fawkes adds imperceptible cloaks to a user’s photos to poison unauthorized facial recognition models, achieving high protection rates across transfer learning and from-scratch training. It shows 95+% protection in typical settings and 80+% when clean images are leaked.

ABSTRACT

Today's proliferation of powerful facial recognition systems poses a real threat to personal privacy. As Clearview.ai demonstrated, anyone can canvas the Internet for data and train highly accurate facial recognition models of individuals without their knowledge. We need tools to protect ourselves from potential misuses of unauthorized facial recognition systems. Unfortunately, no practical or effective solutions exist. In this paper, we propose Fawkes, a system that helps individuals inoculate their images against unauthorized facial recognition models. Fawkes achieves this by helping users add imperceptible pixel-level changes (we call them "cloaks") to their own photos before releasing them. When used to train facial recognition models, these "cloaked" images produce functional models that consistently cause normal images of the user to be misidentified. We experimentally demonstrate that Fawkes provides 95+% protection against user recognition regardless of how trackers train their models. Even when clean, uncloaked images are "leaked" to the tracker and used for training, Fawkes can still maintain an 80+% protection success rate. We achieve 100% success in experiments against today's state-of-the-art facial recognition services. Finally, we show that Fawkes is robust against a variety of countermeasures that try to detect or disrupt image cloaks.

Motivation & Objective

• Motivate the need to protect individuals from unauthorized facial recognition models and invasions of privacy.
• Propose an image cloaking system (Fawkes) that minimally distorts photos while degrading a tracker’s model performance.
• Demonstrate robustness of cloaks against transfer learning, from-scratch training, and countermeasures.

Proposed method

• Compute image-specific cloaks by perturbing each photo within a perceptual budget to maximize feature-space deviation from the original image.
• Use a target class T and a public dataset to guide cloak generation via a landmark-based optimization in feature space.
• Optimize cloak perturbations to minimize the distance between the cloaked image’s features and those of a distant target class, while constraining perceptual distortion (DSSIM).
• Apply a penalty method to enforce the perturbation budget and keep cloaks visually imperceptible.
• Select a target class by maximizing the minimum feature-space distance between the user’s images and the class centroids.
• Evaluate cloaks under scenarios where the tracker uses the same, different, or no pre-trained feature extractor, including transfer learning and training from scratch.

Experimental results

Research questions

• RQ1 Can individuals reliably prevent unauthorized models from recognizing them when cloaks are applied to their online photos?
• RQ2 Do cloaks transfer across different feature extractors and model training methodologies (transfer learning vs. training from scratch)?
• RQ3 How robust are cloaks against countermeasures that detect or disrupt cloaks?
• RQ4 What is the impact of leaking some uncloaked images on privacy protection?
• RQ5 How effective is Fawkes against contemporary commercial facial recognition services?

Key findings

• Cloaking achieves over 95% protection against user recognition regardless of tracker training method.
• When clean (uncloaked) images are leaked, cloaking still yields 80+% protection.
• Cloaks achieve 100% protection against Microsoft Azure Face API, Amazon Rekognition, and Face++ in evaluated setups.
• Cloaks remain robust under various countermeasures aimed at detection or disruption of cloaks.
• Across multiple feature extractors and datasets, cloaks tolerate perceptual budgets up to DSSIM ~0.007 with negligible visual distortion.