[Paper Review] TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems

Wenbo Guo, Lun Wang+3 more|arXiv (Cornell University)|Aug 2, 2019

Adversarial Robustness in Machine LearningComputer Science47 references134 citations

TL;DR

TABOR reframes trojan backdoor inspection as non-convex optimization with new regularization and a fidelity-focused trigger restoration metric, improving detection and restoration over Neural Cleanse under varying trigger conditions.

ABSTRACT

A trojan backdoor is a hidden pattern typically implanted in a deep neural network. It could be activated and thus forces that infected model behaving abnormally only when an input data sample with a particular trigger present is fed to that model. As such, given a deep neural network model and clean input samples, it is very challenging to inspect and determine the existence of a trojan backdoor. Recently, researchers design and develop several pioneering solutions to address this acute problem. They demonstrate the proposed techniques have a great potential in trojan detection. However, we show that none of these existing techniques completely address the problem. On the one hand, they mostly work under an unrealistic assumption (e.g. assuming availability of the contaminated training database). On the other hand, the proposed techniques cannot accurately detect the existence of trojan backdoors, nor restore high-fidelity trojan backdoor images, especially when the triggers pertaining to the trojan vary in size, shape and position. In this work, we propose TABOR, a new trojan detection technique. Conceptually, it formalizes a trojan detection task as a non-convex optimization problem, and the detection of a trojan backdoor as the task of resolving the optimization through an objective function. Different from the existing technique also modeling trojan detection as an optimization problem, TABOR designs a new objective function--under the guidance of explainable AI techniques as well as heuristics--that could guide optimization to identify a trojan backdoor in a more effective fashion. In addition, TABOR defines a new metric to measure the quality of a trojan backdoor identified. Using an anomaly detection method, we show the new metric could better facilitate TABOR to identify intentionally injected triggers in an infected model and filter out false alarms......

Motivation & Objective

• Motivate robust trojan backdoor detection without access to training data or model internals.
• Develop an optimization-based detection framework with regularization to reduce false alarms.
• Propose a trigger restoration metric and technique to accurately recover trojan triggers.
• Evaluate TABOR against existing methods across diverse models and backdoor configurations.
• Show robustness of TABOR to variations in trojan insertion techniques and model complexity.

Proposed method

• Formulate trojan detection as a non-convex optimization problem over a mask M and trigger Delta.
• Introduce four regularization terms to penalize overly large, scattered triggers and to discourage blocking, overlaying, and irrelevant features.
• Design R1 and R2 regularizations to reduce the adversarial subspace and encourage concise, contiguous triggers.
• Add R3 regularization to avoid blocking key image features and maintain correct classification after trigger removal.
• Incorporate R4 regularization inspired by explainable AI to refine trigger fidelity through feature importance insights.
• Solve the optimization with a tailored approach guided by observations of false alarms and trigger overlays.

Experimental results

Research questions

• RQ1 Can TABOR reliably detect the presence of a trojan backdoor without access to training data or model internals?
• RQ2 Does TABOR accurately restore high-fidelity trojan triggers across varying trigger shapes, sizes, and positions?
• RQ3 How does TABOR perform relative to Neural Cleanse under diverse trojan configurations and model complexities?
• RQ4 Can the regularization-guided objective reduce false alarms and improve trigger fidelity in both infected and clean models?

Key findings

• TABOR shows improved detection performance and trigger restoration fidelity compared with the state-of-the-art Neural Cleanse.
• Regularization terms reduce the adversarial subspace and suppress false alarms from scattered or oversized triggers.
• A blocking-trigger regularization eliminates triggers that obstruct key image content.
• An overlaying-trigger regularization helps extract a higher-fidelity representation of the intended trojan trigger.
• The explainable-AI-inspired regularization refines the restored trigger by pruning irrelevant features.