[Paper Review] Defending Neural Backdoors via Generative Distribution Modeling

Ximing Qiao, Yukun Yang+1 more|arXiv (Cornell University)|Oct 10, 2019

Adversarial Robustness in Machine LearningComputer Science65 citations

TL;DR

This paper proposes MESA, a max-entropy staircase approximator, to model the unknown valid backdoor trigger distribution and use it to robustly defend neural backdoors.

ABSTRACT

Neural backdoor attack is emerging as a severe security threat to deep learning, while the capability of existing defense methods is limited, especially for complex backdoor triggers. In the work, we explore the space formed by the pixel values of all possible backdoor triggers. An original trigger used by an attacker to build the backdoored model represents only a point in the space. It then will be generalized into a distribution of valid triggers, all of which can influence the backdoored model. Thus, previous methods that model only one point of the trigger distribution is not sufficient. Getting the entire trigger distribution, e.g., via generative modeling, is a key to effective defense. However, existing generative modeling techniques for image generation are not applicable to the backdoor scenario as the trigger distribution is completely unknown. In this work, we propose max-entropy staircase approximator (MESA), an algorithm for high-dimensional sampling-free generative modeling and use it to recover the trigger distribution. We also develop a defense technique to remove the triggers from the backdoored model. Our experiments on Cifar10/100 dataset demonstrate the effectiveness of MESA in modeling the trigger distribution and the robustness of the proposed defense method.

Motivation & Objective

• Motivate and formalize the problem that backdoor triggers form a continuous distribution in pixel space, not a single point.
• Propose a sampling-free generative modeling approach (MESA) to recover the valid trigger distribution without direct sampling.
• Develop a defense pipeline that uses the recovered trigger distribution to retrain and remove backdoors.
• Demonstrate robustness and effectiveness of the method across multiple trigger types on CIFAR-10/100.

Proposed method

• Introduce the max-entropy staircase approximator (MESA) that ensembles N sub-models to approximate the unknown trigger distribution.
• Formulate the problem as sampling-free generation where an unknown distribution f over triggers is learned via an upper-bounded density and a surrogate ASR-based testing function F.
• Use entropy maximization via a mutual information neural estimator (MINE) to train sub-models G_theta_i under staircase thresholds beta_i.
• Ensemble sub-models with weights gamma_i derived from entropy to approximate the target distribution F and reconstruct the trigger distribution f.
• Provide Algorithm 2 for implementing MESA to model valid triggers and Algorithm 1 for the MESA ensembling mechanism.
• Define a three-step defense: detect the attacked class, retrain with triggers drawn from the modeled distribution, and evaluate/adjust beta_i for best defense.

Experimental results

Research questions

• RQ1 Can backdoor trigger distributions be effectively modeled as a distribution in pixel space rather than a single trigger point?
• RQ2 Can max-entropy staircase approximators (MESA) recover the valid trigger distribution without sampling from it?
• RQ3 Does defense based on the recovered trigger distribution robustly reduce ASR across diverse triggers compared to single-reverse-trigger baselines?
• RQ4 How do hyperparameters alpha and beta_i affect the quality of the modeled distribution and defense performance?
• RQ5 Is the proposed distribution-based defense effective across different CIFAR datasets and target classes?

Key findings

• MESA can model the valid trigger distribution for various 3x3 and color triggers on CIFAR-10/100.
• Defense based on the modeled trigger distribution reduces the original trigger ASR from 92.3%–99.8% to 1.2%–5.9%.
• Baseline defense using a single reversed trigger shows much higher variance and worse worst-case ASR (up to 51% in some runs).
• The targeted class detection achieved by MESA reliably distinguishes attacked vs. non-attacked classes (high ASR for true target class, low for others).
• Ensembling with beta_i=0.9 yields best defense performance with after-defense ASR around 3.4% on average (worst-case 5.9%).
• Defense robustness is demonstrated across both black-white triggers and random-color triggers on CIFAR-10/100.