[Paper Review] LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition

Valeriia Cherepanova, Micah Goldblum+5 more|arXiv (Cornell University)|May 3, 2021

Adversarial Robustness in Machine Learning20 references52 citations

TL;DR

LowKey proposes a practical adversarial defense that degrades commercial facial recognition APIs like Amazon Rekognition and Microsoft Azure Face by applying image-level perturbations tailored to the full processing pipeline. It achieves below 1% accuracy on both systems, demonstrating effectiveness against real-world industrial deployments.

ABSTRACT

Facial recognition systems are increasingly deployed by private corporations, government agencies, and contractors for consumer services and mass surveillance programs alike. These systems are typically built by scraping social media profiles for user images. Adversarial perturbations have been proposed for bypassing facial recognition systems. However, existing methods fail on full-scale systems and commercial APIs. We develop our own adversarial filter that accounts for the entire image processing pipeline and is demonstrably effective against industrial-grade pipelines that include face detection and large scale databases. Additionally, we release an easy-to-use webtool that significantly degrades the accuracy of Amazon Rekognition and the Microsoft Azure Face Recognition API, reducing the accuracy of each to below 1%.

Motivation & Objective

• To address the growing threat of mass surveillance via facial recognition systems trained on social media data.
• To develop an adversarial defense effective against full-scale industrial pipelines, not just academic benchmarks.
• To create a practical, user-friendly tool that enables individuals to protect their privacy on social media platforms.
• To demonstrate that adversarial attacks can be effective against real-world commercial APIs despite robust preprocessing and large-scale databases.

Proposed method

• Design a custom adversarial filter that models the entire image processing pipeline, including face detection and recognition stages.
• Optimize perturbations to be robust across multiple stages of image transformation, such as resizing, cropping, and normalization.
• Train and deploy the filter to generate subtle, imperceptible image modifications that degrade recognition performance.
• Integrate the filter into a web-based tool for public use, enabling users to upload images and receive adversarially perturbed versions.
• Test the method against industrial-grade APIs, including Amazon Rekognition and Microsoft Azure Face, under realistic conditions.
• Ensure the perturbations remain effective even after common image compression and transmission processes.

Experimental results

Research questions

• RQ1 Can adversarial perturbations be effective against full-scale commercial facial recognition systems, including those with robust preprocessing and large-scale databases?
• RQ2 How does the performance of adversarial attacks degrade when applied to real-world APIs like Amazon Rekognition and Microsoft Azure Face?
• RQ3 Can a user-friendly webtool effectively protect social media users from facial recognition without requiring technical expertise?
• RQ4 What level of degradation in recognition accuracy can be achieved using a pipeline-aware adversarial approach?
• RQ5 How resilient are the adversarial perturbations to common image transformations in social media workflows?

Key findings

• The adversarial filter reduced the accuracy of Amazon Rekognition to below 1% on tested images.
• The same filter degraded Microsoft Azure Face Recognition API performance to under 1% accuracy.
• The method remains effective even after image compression and common web transmission formats.
• The webtool enables non-technical users to apply adversarial perturbations to their social media images with minimal effort.
• The approach is robust across multiple stages of the facial recognition pipeline, including face detection and embedding extraction.
• The results demonstrate that industrial-grade facial recognition systems are vulnerable to well-crafted, pipeline-aware adversarial attacks.