[Paper Review] Attacking CNN-based anti-spoofing face authentication in the physical domain

Bowen Zhang, Benedetta Tondi+1 more|arXiv (Cornell University)|Oct 1, 2019

Biometric Identification and Security29 references4 citations

TL;DR

This paper proposes a physical-domain adversarial attack against CNN-based anti-spoofing face authentication systems by pre-emptively compensating for rebroadcast distortions. The method generates adversarial images that evade spoofing detection, pass face detection, and trick face recognition, achieving 97.04% impersonation success rate in multi-trial scenarios on the REPLAY-MOBILE dataset.

ABSTRACT

In this paper, we study the vulnerability of anti-spoofing methods based on deep learning against adversarial perturbations. We first show that attacking a CNN-based anti-spoofing face authentication system turns out to be a difficult task. When a spoofed face image is attacked in the physical world, in fact, the attack has not only to remove the rebroadcast artefacts present in the image, but it has also to take into account that the attacked image will be recaptured again and then compensate for the distortions that will be re-introduced after the attack by the subsequent rebroadcast process. Subsequently, we propose a method to craft robust physical domain adversarial images against anti-spoofing CNN-based face authentication. The attack built in this way can successfully pass all the steps in the authentication chain (that is, face detection, face recognition and spoofing detection), by achieving simultaneously the following goals: i) make the spoofing detection fail; ii) let the facial region be detected as a face and iii) recognized as belonging to the victim of the attack. The effectiveness of the proposed attack is validated experimentally within a realistic setting, by considering the REPLAY-MOBILE database, and by feeding the adversarial images to a real face authentication system capturing the input images through a mobile phone camera.

Motivation & Objective

• Address the challenge of crafting physical-domain adversarial examples that bypass CNN-based anti-spoofing mechanisms in real-world face authentication systems.
• Overcome the difficulty that rebroadcast processes (e.g., printing and recapturing) introduce distortions that can expose adversarial patterns.
• Ensure adversarial images successfully pass all stages of the authentication pipeline: face detection, face recognition, and spoofing detection.
• Validate the attack under realistic conditions using a mobile phone camera and the REPLAY-MOBILE database.
• Demonstrate the feasibility of adversarial spoofing even when the system is designed to detect physical presentation attacks.

Proposed method

• Model the rebroadcast process as a distortion channel that transforms digital adversarial images into physical ones, and invert this process to pre-emptively compensate for introduced artifacts.
• Craft adversarial perturbations on a digital image that has already undergone a simulated rebroadcast process (i.e., image $I_s$), rather than on the original image $I_0$.
• Use a differentiable image transformation pipeline to simulate the physical capture process, enabling end-to-end optimization of adversarial examples.
• Optimize the adversarial example to minimize the loss in spoofing detection while maintaining high confidence in face detection and recognition modules.
• Apply the attack to multiple source images of the same identity to increase the chance of success under multiple access attempts.
• Define and compute the Impersonate Success Rate (ISR) using a probabilistic model assuming three random attempts per identity.

Experimental results

Research questions

• RQ1 Can adversarial examples be crafted in the physical domain to successfully bypass CNN-based anti-spoofing mechanisms in face authentication systems?
• RQ2 How can the distortions introduced during the rebroadcast process (e.g., printing and recapturing) be modeled and compensated for in adversarial attack generation?
• RQ3 To what extent can adversarial spoofing images pass all stages of the face authentication pipeline—face detection, recognition, and spoofing detection—simultaneously?
• RQ4 What is the success rate of such attacks when multiple attempts are allowed, and how does it vary across different identities?
• RQ5 Can the attack remain effective when the adversarial image is captured via a mobile phone camera in a realistic deployment setting?

Key findings

• The proposed attack achieves a 97.04% Impersonate Success Rate (ISR) across all identities in Set#1 and 95.82% in Set#2, indicating high reliability under multiple access attempts.
• In the face detection stage, the attack fails in 0% of cases across both sets, confirming that the adversarial images are reliably detected as faces.
• Face recognition fails in 7.14% of cases in Set#1 and 5% in Set#2, meaning the system often misidentifies the victim as someone else, but the attack still succeeds overall.
• Spoofing detection fails in 92.86% of cases in Set#1 and 95% in Set#2, demonstrating that the adversarial images successfully evade anti-spoofing detection.
• The attack remains effective even when the adversarial image is post-processed with Photoshop, indicating robustness to common image manipulations.
• The worst-case ISR across identities is 80.7%, achieved for Client08, while seven out of nine identities achieve 100% ISR, showing strong overall performance.