[Paper Review] Deep Partition Aggregation: Provable Defenses against General Poisoning Attacks

Alexander Levine, Soheil Feizi|arXiv (Cornell University)|May 3, 2021

Adversarial Robustness in Machine Learning40 references22 citations

TL;DR

This paper proposes Deep Partition Aggregation (DPA) and Semi-Supervised DPA (SS-DPA), novel certified defenses against general and label-flipping poisoning attacks, respectively. DPA uses hash-based data partitioning and ensemble learning to provide provable robustness, achieving state-of-the-art certification: over 500 poison insertions on MNIST and nine on CIFAR-10, with SS-DPA tolerating over 600 label flips on MNIST and 300 on CIFAR-10.

ABSTRACT

Adversarial poisoning attacks distort training data in order to corrupt the test-time behavior of a classifier. A provable defense provides a certificate for each test sample, which is a lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample's classification. We propose two novel provable defenses against poisoning attacks: (i) Deep Partition Aggregation (DPA), a certified defense against a general poisoning threat model, defined as the insertion or deletion of a bounded number of samples to the training set --- by implication, this threat model also includes arbitrary distortions to a bounded number of images and/or labels; and (ii) Semi-Supervised DPA (SS-DPA), a certified defense against label-flipping poisoning attacks. DPA is an ensemble method where base models are trained on partitions of the training set determined by a hash function. DPA is related to both subset aggregation, a well-studied ensemble method in classical machine learning, as well as to randomized smoothing, a popular provable defense against evasion (inference) attacks. Our defense against label-flipping poison attacks, SS-DPA, uses a semi-supervised learning algorithm as its base classifier model: each base classifier is trained using the entire unlabeled training set in addition to the labels for a partition. SS-DPA significantly outperforms the existing certified defense for label-flipping attacks (Rosenfeld et al., 2020) on both MNIST and CIFAR-10: provably tolerating, for at least half of test images, over 600 label flips (vs. < 200 label flips) on MNIST and over 300 label flips (vs. 175 label flips) on CIFAR-10. Against general poisoning attacks where no prior certified defenses exists, DPA can certify ≥ 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10. These results establish new state-of-the-art provable defenses against general and label-flipping poison attacks.

Motivation & Objective

• To develop a provable defense against general poisoning attacks, including sample insertion, deletion, and label/image distortion.
• To design a certified defense specifically for label-flipping poisoning attacks, outperforming prior work in robustness.
• To establish a new state-of-the-art in certified robustness for both general and label-flipping poisoning threats.
• To leverage ensemble learning and semi-supervised training to improve certification guarantees while maintaining high accuracy.
• To provide a theoretical certificate for each test sample, quantifying the minimum poisoning magnitude required to corrupt predictions.

Proposed method

• DPA partitions the training set using a random hash function, training base models on disjoint subsets to form an ensemble.
• The final prediction is determined by majority vote across base models, with robustness certified via a lower bound on the number of samples needed to alter the prediction.
• SS-DPA extends DPA by using semi-supervised learning: each base model is trained on the full unlabeled data and labels from its partition, improving label-flip robustness.
• Certification relies on a combinatorial argument that bounds the number of label or data perturbations required to change the ensemble prediction.
• The method inherits properties from both subset aggregation and randomized smoothing, combining their strengths for poisoning defense.
• Robustness certificates are computed efficiently using the structure of the partitioning and voting mechanism, enabling practical deployment.

Experimental results

Research questions

• RQ1 Can a certified defense be constructed that provably resists general poisoning attacks involving insertion, deletion, or distortion of a bounded number of training samples?
• RQ2 Can a semi-supervised variant of the defense significantly improve robustness against label-flipping attacks compared to existing certified defenses?
• RQ3 What is the maximum number of poisoning perturbations (insertions or label flips) that can be certified for a given test sample using this framework?
• RQ4 How does the performance of the proposed defense compare to prior certified defenses on standard benchmarks like MNIST and CIFAR-10?
• RQ5 Can the ensemble structure of DPA be leveraged to provide stronger theoretical guarantees than existing methods under the same threat model?

Key findings

• DPA certifies robustness for at least 50% of test images on MNIST against over 500 poison image insertions, setting a new state-of-the-art.
• On CIFAR-10, DPA provides certification against nine poison insertions for at least half of the test images, demonstrating strong general poisoning resilience.
• SS-DPA outperforms prior certified defenses on label-flipping attacks, tolerating over 600 label flips on MNIST (vs. <200 in prior work).
• On CIFAR-10, SS-DPA certifies robustness against over 300 label flips (vs. 175 in prior work), establishing a new benchmark.
• The proposed defenses achieve significantly higher certified robustness than existing methods under both general and label-flipping poisoning threat models.
• The combination of partitioning, ensemble voting, and semi-supervised learning enables both strong theoretical guarantees and practical performance on standard datasets.