[Paper Review] Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

Cong Liao, Haoti Zhong+3 more|arXiv (Cornell University)|Aug 30, 2018

Adversarial Robustness in Machine Learning41 references106 citations

TL;DR

The paper presents two stealthy backdoor injection methods for CNNs in image classification, enabling targeted misclassification with minimal accuracy loss and low poisoning rate.

ABSTRACT

Deep learning models have consistently outperformed traditional machine learning models in various classification tasks, including image classification. As such, they have become increasingly prevalent in many real world applications including those where security is of great concern. Such popularity, however, may attract attackers to exploit the vulnerabilities of the deployed deep learning models and launch attacks against security-sensitive applications. In this paper, we focus on a specific type of data poisoning attack, which we refer to as a {\em backdoor injection attack}. The main goal of the adversary performing such attack is to generate and inject a backdoor into a deep learning model that can be triggered to recognize certain embedded patterns with a target label of the attacker's choice. Additionally, a backdoor injection attack should occur in a stealthy manner, without undermining the efficacy of the victim model. Specifically, we propose two approaches for generating a backdoor that is hardly perceptible yet effective in poisoning the model. We consider two attack settings, with backdoor injection carried out either before model training or during model updating. We carry out extensive experimental evaluations under various assumptions on the adversary model, and demonstrate that such attacks can be effective and achieve a high attack success rate (above $90\%$) at a small cost of model accuracy loss (below $1\%$) with a small injection rate (around $1\%$), even under the weakest assumption wherein the adversary has no knowledge either of the original training data or the classifier model.

Motivation & Objective

• Motivate the study of backdoor injection attacks in CNN-based image classification and assess security risks in security-sensitive applications.
• Propose two backdoor generation strategies that are visually imperceptible and effective.
• Evaluate attack feasibility under different attacker knowledge and capability scenarios.
• Demonstrate that high attack success is achievable with low poisoning rates while preserving overall model performance.

Proposed method

• Introduce two backdoor generation strategies: a patterned static perturbation mask and a targeted adaptive perturbation mask.
• Formalize the backdoor injection as data poisoning with an injection dataset D_A added to the training set.
• Use mini-batch gradient descent training with poisoned data to maximize both classification accuracy and backdoor success.
• Attack can be conducted in two settings: Backdoor Injection Before training (BIB) and Backdoor Injection During updating (BID).
• Adaptive perturbation uses a DeepFool-inspired iterative method to push samples toward the decision boundary of the target class under an l_infinity constraint.
• Provide mathematical formulation for poisoning objective and conditions for backdoor effectiveness.

Experimental results

Research questions

• RQ1 How effective are visually stealthy backdoor perturbations at triggering targeted misclassification in CNNs?
• RQ2 What are the limits of backdoor attacks under varying attacker knowledge (FK, PKD, PKM, MK) and capabilities?
• RQ3 Can backdoors be injected with minimal impact on overall test accuracy while maintaining high attack success?
• RQ4 How do two backdoor generation strategies compare in terms of stealthiness and effectiveness?

Key findings

• Attack success rate exceeds 90% in several scenarios with around 1% poisoning rate.
• Classification accuracy loss remains below 1% under the tested conditions.
• Backdoor perturbations can be visually imperceptible and difficult to detect by machine detectors.
• Two backdoor generation approaches (patterned static and targeted adaptive) provide flexible options for stealthy backdoor creation.
• Attack effectiveness is demonstrated under various attacker models, including weak adversaries with no knowledge of original data or model.