[Paper Review] Fawkes: Protecting Personal Privacy against Unauthorized Deep Learning Models

Shawn Shan, Emily Wenger+4 more|arXiv (Cornell University)|Feb 19, 2020

Adversarial Robustness in Machine LearningComputer Science33 references14 citations

TL;DR

Fawkes is a privacy-preserving system that protects individuals from unauthorized facial recognition by injecting imperceptible pixel-level 'cloaks' into personal photos before public release. These cloaks cause trained facial recognition models to misidentify the user, achieving over 95% protection against model inference and 80% success even when clean images are leaked, with 100% effectiveness against state-of-the-art services.

ABSTRACT

Today's proliferation of powerful facial recognition systems poses a real threat to personal privacy. As this http URL demonstrated, anyone can canvas the Internet for data and train highly accurate facial recognition models of individuals without their knowledge. We need tools to protect ourselves from potential misuses of unauthorized facial recognition systems. Unfortunately, no practical or effective solutions exist. In this paper, we propose Fawkes, a system that helps individuals inoculate their images against unauthorized facial recognition models. Fawkes achieves this by helping users add imperceptible pixel-level changes (we call them cloaks) to their own photos before releasing them. When used to train facial recognition models, these cloaked images produce functional models that consistently cause normal images of the user to be misidentified. We experimentally demonstrate that Fawkes provides 95+% protection against user recognition regardless of how trackers train their models. Even when clean, uncloaked images are leaked to the tracker and used for training, Fawkes can still maintain an 80+% protection success rate. We achieve 100% success in experiments against today's state-of-the-art facial recognition services. Finally, we show that Fawkes is robust against a variety of countermeasures that try to detect or disrupt image cloaks.

Motivation & Objective

• To address the growing threat of unauthorized facial recognition models trained on publicly available images.
• To develop a practical, user-controlled defense mechanism that does not require changes to existing facial recognition systems.
• To ensure that cloaked images remain functional for legitimate use while degrading the performance of unauthorized models.
• To maintain robustness against detection and removal of cloaks by adversarial countermeasures.

Proposed method

• Users apply imperceptible pixel-level perturbations, termed 'cloaks,' to their personal images before public sharing.
• Cloaked images are used to train facial recognition models, which then misidentify the user when presented with clean, uncloaked images.
• The cloaking process is designed to be undetectable to human observers and resilient to common image transformations.
• The system ensures that the cloaked images remain semantically valid and usable for normal applications like social media.
• Fawkes leverages model poisoning techniques to embed a backdoor that triggers misclassification only on clean test images of the user.
• The method is robust against various image preprocessing and adversarial detection techniques aimed at removing or identifying cloaks.

Experimental results

Research questions

• RQ1 Can a user protect their privacy by modifying their own images in a way that degrades unauthorized facial recognition models without affecting legitimate use?
• RQ2 How effective is the cloaking technique in maintaining high misidentification rates across diverse training and inference scenarios?
• RQ3 Can Fawkes preserve protection even when attackers use clean, uncloaked images from the same dataset for model training?
• RQ4 How resilient is the cloaking mechanism against detection and removal attempts by adversaries?
• RQ5 Does Fawkes achieve strong performance against state-of-the-art commercial facial recognition services?

Key findings

• Fawkes achieves over 95% protection against user recognition across all tested facial recognition model training scenarios.
• Even when attackers include clean, uncloaked images in their training data, Fawkes maintains an 80%+ success rate in misidentifying the user.
• The system demonstrates 100% effectiveness in causing misidentification when tested against today's state-of-the-art facial recognition services.
• Fawkes remains robust against various countermeasures designed to detect or remove the cloaking patterns.
• The cloaking technique is imperceptible to humans and does not degrade the visual quality or usability of the images.