[Paper Review] Backdoor Attacks on Facial Recognition in the Physical World

Emily Wenger, Josephine Passananti+3 more|arXiv (Cornell University)|Jun 25, 2020

Adversarial Robustness in Machine Learning34 references24 citations

TL;DR

This paper investigates the feasibility of physical backdoor attacks on facial recognition systems using real-world accessories as triggers. It demonstrates that while physical triggers are less effective and more prone to false positives than digital ones, they remain a serious threat due to the failure of existing defenses, which rely on assumptions invalid in physical settings.

ABSTRACT

Backdoor attacks embed hidden malicious behaviors inside deep neural networks (DNNs) that are only activated when a specific is present on some input to the model. A variety of these attacks have been successfully proposed and evaluated, generally using digitally generated patterns or images as triggers. Despite significant prior work on the topic, a key question remains unanswered: can backdoor attacks be physically realized in the real world, and what limitations do attackers face in executing them? In this paper, we present results of a detailed study on DNN backdoor attacks in the physical world, specifically focused on the task of facial recognition. We take 3205 photographs of 10 volunteers in a variety of settings and backgrounds and train a facial recognition model using transfer learning from VGGFace. We evaluate the effectiveness of 9 accessories as potential triggers, and analyze impact from external factors such as lighting and image quality. First, we find that triggers vary significantly in efficacy and a key factor is that facial recognition models are heavily tuned to features on the face and less so to features around the periphery. Second, the efficacy of most trigger objects is. negatively impacted by lower image quality but unaffected by lighting. Third, most triggers suffer from false positives, where non-trigger objects unintentionally activate the backdoor. Finally, we evaluate 4 backdoor defenses against physical backdoors. We show that they all perform poorly because physical triggers break key assumptions they made based on triggers in the digital domain. Our key takeaway is that implementing physical backdoors is much more challenging than described in literature for both attackers and defenders and much more work is necessary to understand how backdoors work in the real world.

Motivation & Objective

• To evaluate the real-world feasibility of backdoor attacks on facial recognition systems using physical triggers.
• To assess how environmental factors like lighting and image quality affect trigger efficacy.
• To analyze the performance of existing backdoor defenses against physical triggers.
• To identify key limitations faced by attackers and defenders in deploying and detecting physical backdoors.

Proposed method

• Collected 3,205 real-world facial images of 10 volunteers under varied settings and backgrounds.
• Trained a facial recognition model using transfer learning from VGGFace on the collected dataset.
• Evaluated 9 physical accessories as potential triggers, testing their success rate under different lighting and image quality conditions.
• Measured backdoor success rate and false positive rates across diverse physical trigger objects.
• Assessed four existing backdoor defenses against physical triggers to determine their robustness.
• Analyzed model sensitivity to facial features versus peripheral features, explaining variations in trigger effectiveness.

Experimental results

Research questions

• RQ1 Can physical triggers successfully activate backdoors in real-world facial recognition systems?
• RQ2 How do lighting conditions and image quality impact the success rate of physical triggers?
• RQ3 To what extent do physical triggers cause false positives when non-trigger objects are present?
• RQ4 Why do existing digital-domain backdoor defenses fail when applied to physical triggers?
• RQ5 What role do facial feature localization and peripheral feature sensitivity play in trigger efficacy?

Key findings

• Physical triggers are significantly less effective than digital triggers, with efficacy varying widely across different accessories.
• Facial recognition models are more sensitive to features on the face than to peripheral features, making facial-based triggers more effective.
• Lower image quality negatively impacts trigger efficacy, while lighting conditions have minimal effect.
• Most physical triggers suffer from high false positive rates, where non-trigger objects unintentionally activate the backdoor.
• All tested backdoor defenses fail against physical triggers because they rely on assumptions that do not hold in physical-world scenarios.
• The study concludes that physical backdoors are more challenging to implement and detect than previously assumed, requiring new research approaches.