[论文解读] Invisible Mask: Practical Attacks on Face Recognition with Infrared

Zhe Zhou, Di Tang+4 more|arXiv (Cornell University)|Mar 13, 2018

Adversarial Robustness in Machine Learning参考文献 26被引用 70

一句话总结

论文提出一种基于红外的对抗攻击，利用安装在帽子上的未见的红外LED灯，在脸部制造看不见的扰动，使其能够在如 FaceNet 的面部识别系统上躲避监控并进行身份冒充。

ABSTRACT

Accurate face recognition techniques make a series of critical applications possible: policemen could employ it to retrieve criminals' faces from surveillance video streams; cross boarder travelers could pass a face authentication inspection line without the involvement of officers. Nonetheless, when public security heavily relies on such intelligent systems, the designers should deliberately consider the emerging attacks aiming at misleading those systems employing face recognition. We propose a kind of brand new attack against face recognition systems, which is realized by illuminating the subject using infrared according to the adversarial examples worked out by our algorithm, thus face recognition systems can be bypassed or misled while simultaneously the infrared perturbations cannot be observed by raw eyes. Through launching this kind of attack, an attacker not only can dodge surveillance cameras. More importantly, he can impersonate his target victim and pass the face authentication system, if only the victim's photo is acquired by the attacker. Again, the attack is totally unobservable by nearby people, because not only the light is invisible, but also the device we made to launch the attack is small enough. According to our study on a large dataset, attackers have a very high success rate with a over 70\% success rate for finding such an adversarial example that can be implemented by infrared. To the best of our knowledge, our work is the first one to shed light on the severity of threat resulted from infrared adversarial examples against face recognition.

研究动机与目标

• Motivate and quantify practical risks of infrared adversarial perturbations to face recognition systems.
• Develop an algorithm to generate IR LED-based adversarial examples under realistic hardware constraints.
• Design and calibrate a small, inconspicuous device to implement the adversarial perturbations in real-world settings.
• Evaluate the effectiveness of the attack on a real FR system (FaceNet) and on a large-scale face dataset (LFW).

提出的方法

• Model IR LED light spots as controllable perturbations with position, size, brightness, and color characteristics.
• Formulate an optimization to minimize the distance between the attacker's embedding and the victim's embedding under IR LED constraints.
• Synthesize adversarial examples by summing modeled light spots onto the attacker’s image and converting to a purple IR-visible perturbation.
• Design a physical device with three IR LEDs on a cap peak, plus calibration tools to align the perturbation with the generated adversarial example.
• Implement a calibration workflow that compares images with LEDs off and on to adjust spot centers and brightness to minimize embedding distance.
• Apply an Adam optimizer to minimize the loss J(f(I_syn), f(I_vtm)) with respect to spot parameters and overall amplification.

实验结果

研究问题

• RQ1 Can infrared-based perturbations be crafted to reliably bypass or impersonate targets in FR systems under realistic hardware constraints?
• RQ2 How effective is an IR-based invisible mask against a mainstream FR model (FaceNet) and what is the achievable success rate for dodging and impersonation?
• RQ3 What is the role of calibration and manual fine-tuning in realizing practical adversarial examples with IR LEDs?
• RQ4 What are the practical considerations (threat model, device design, safety) for deploying such an IR-based attack in the wild?

主要发现

• Dodging attacks achieved a 100% success rate in the reported physical test.
• Impersonation success probability for finding actionable adversarial examples is over 70% on large-scale study over LFW data.
• A cap-based device with three 850 nm IR LEDs is sufficient to disrupt landmark pre-processing and enable both dodging and impersonation.
• An optimization-based pipeline can generate IR LED layouts that approximate imperceptible perturbations realizable by hardware.
• Calibration and fine-tuning significantly improve attack success, with manual adjustments sometimes yielding better results than raw optimization alone.