[Paper Review] Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review

Yansong Gao, Bao Gia Doan+6 more|arXiv (Cornell University)|Jul 21, 2020

Adversarial Robustness in Machine Learning180 references132 citations

TL;DR

This paper provides a systematic taxonomy of backdoor attack surfaces in deep learning and surveys existing attacks and countermeasures, evaluating their advantages and limitations. It also discusses the flip side and future research directions.

ABSTRACT

This work provides the community with a timely comprehensive review of backdoor attacks and countermeasures on deep learning. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures, and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which are explored for i) protecting intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor.Overall, the research on defense is far behind the attack, and there is no single defense that can prevent all types of backdoor attacks. In some cases, an attacker can intelligently bypass existing defenses with an adaptive attack. Drawing the insights from the systematic review, we also present key areas for future research on the backdoor, such as empirical security evaluations from physical trigger attacks, and in particular, more efficient and practical countermeasures are solicited.

Motivation & Objective

• Provide a taxonomy of backdoor attack surfaces based on attacker capabilities and ML pipeline stage.
• Catalogue and compare backdoor attacks across surfaces and assess their strengths and limitations.
• Summarize countermeasures and categorize them by deployment stage and data/model focus.
• Discuss the practical implications, flip side applications, and future research directions.

Proposed method

• Define and formalize backdoor attack concepts and metrics such as CDA and ASR.
• Systematically categorize attack surfaces into six classes: code poisoning, outsourcing, pretrained, data collection, collaborative learning, and post-deployment.
• Review and summarize representative attacks under each surface with qualitative comparisons.
• Categorize countermeasures into blind removal, offline inspection, online inspection, and post backdoor removal, and compare their pros/cons.
• Discuss broader implications, including IP protection, honeypots, and data deletion verification, and outline future research directions.

Experimental results

Research questions

• RQ1 What taxonomy best captures the surfaces through which backdoor attacks can be executed in DL pipelines?
• RQ2 What are the major backdoor attack techniques within each surface, and how do they compare in capability and performance?
• RQ3 What defense strategies exist, how are they categorized, and what are their limitations against adaptive attacks?
• RQ4 What are the broader implications and potential positive applications (flip side) of backdoor research?
• RQ5 What are the key open challenges and future directions for empirical evaluation and defense development?

Key findings

• Backdoor attacks can be organized into six surfaces corresponding to stages in the ML pipeline and attacker capabilities.
• Attacks maintain normal performance on clean data while achieving high attack success rates when triggered.
• There is no single defense that prevents all backdoor variants, and adaptive attackers can bypass some defenses.
• Defensive research lags behind offensive techniques, highlighting the need for practical and efficient countermeasures.
• The review identifies broader uses of backdoor research, such as protecting IP, acting as honeypots, and verifying data deletion.
• The authors propose directions for future work, including empirical security evaluations with physical triggers and more effective defenses.