[论文解读] Backdoor Attacks on Facial Recognition in the Physical World

Emily Wenger, Josephine Passananti+3 more|arXiv (Cornell University)|Jun 25, 2020

Adversarial Robustness in Machine Learning参考文献 34被引用 24

一句话总结

本文研究了使用现实世界配饰作为触发器对人脸识别系统实施物理后门攻击的可行性。结果表明，尽管物理触发器在有效性方面不如数字触发器，且更容易产生误报，但由于现有防御机制在物理环境下的假设不成立，其仍构成严重威胁。

ABSTRACT

Backdoor attacks embed hidden malicious behaviors inside deep neural networks (DNNs) that are only activated when a specific is present on some input to the model. A variety of these attacks have been successfully proposed and evaluated, generally using digitally generated patterns or images as triggers. Despite significant prior work on the topic, a key question remains unanswered: can backdoor attacks be physically realized in the real world, and what limitations do attackers face in executing them? In this paper, we present results of a detailed study on DNN backdoor attacks in the physical world, specifically focused on the task of facial recognition. We take 3205 photographs of 10 volunteers in a variety of settings and backgrounds and train a facial recognition model using transfer learning from VGGFace. We evaluate the effectiveness of 9 accessories as potential triggers, and analyze impact from external factors such as lighting and image quality. First, we find that triggers vary significantly in efficacy and a key factor is that facial recognition models are heavily tuned to features on the face and less so to features around the periphery. Second, the efficacy of most trigger objects is. negatively impacted by lower image quality but unaffected by lighting. Third, most triggers suffer from false positives, where non-trigger objects unintentionally activate the backdoor. Finally, we evaluate 4 backdoor defenses against physical backdoors. We show that they all perform poorly because physical triggers break key assumptions they made based on triggers in the digital domain. Our key takeaway is that implementing physical backdoors is much more challenging than described in literature for both attackers and defenders and much more work is necessary to understand how backdoors work in the real world.

研究动机与目标

• 评估使用物理触发器在真实世界中对人脸识别系统实施后门攻击的可行性。
• 评估光照和图像质量等环境因素对触发器有效性的影响。
• 分析现有后门防御机制在应对物理触发器时的表现。
• 识别攻击者和防御者在部署和检测物理后门时面临的关键局限性。

提出的方法

• 在不同场景和背景条件下，收集了10名志愿者的3,205张真实世界人脸图像。
• 基于VGGFace的迁移学习，在收集的数据集上训练了一个人脸识别模型。
• 评估了9种物理配饰作为潜在触发器，测试其在不同光照和图像质量条件下的成功率。
• 在多种物理触发器对象上测量了后门成功率和误报率。
• 评估了四种现有后门防御机制对物理触发器的鲁棒性。
• 分析了模型对人脸特征与周边特征的敏感性，以解释触发器有效性差异的原因。

实验结果

研究问题

• RQ1 物理触发器能否在真实世界的人脸识别系统中成功激活后门？
• RQ2 光照条件和图像质量如何影响物理触发器的成功率？
• RQ3 当存在非触发物体时，物理触发器导致误报的程度如何？
• RQ4 为何现有数字领域后门防御机制在应用于物理触发器时会失效？
• RQ5 人脸特征定位与周边特征敏感性在触发器有效性中起到何种作用？

主要发现

• 物理触发器的有效性显著低于数字触发器，且不同配饰之间的有效性差异显著。
• 人脸识别模型对脸部特征的敏感性高于对周边特征的敏感性，因此基于面部的触发器更有效。
• 较低的图像质量会负面影响触发器的有效性，而光照条件的影响较小。
• 大多数物理触发器存在较高的误报率，即非触发物体会无意中激活后门。
• 所有测试的后门防御机制均无法抵御物理触发器，因为它们依赖于在物理世界中不成立的假设。
• 本研究得出结论：与以往假设相比，物理后门的部署和检测更具挑战性，亟需新的研究方法。