[Paper Review] Detection of Adversarial Training Examples in Poisoning Attacks through Anomaly Detection

Andrea Paudice, Luis Muñoz-González+2 more|arXiv (Cornell University)|Feb 8, 2018

Anomaly Detection Techniques and Applications28 references112 citations

TL;DR

The paper proposes using outlier detection on a pre-filtered training set to detect adversarial poisoning examples in linear classifiers, and evaluates its effectiveness against optimal poisoning attacks.

ABSTRACT

Machine learning has become an important component for many systems and applications including computer vision, spam filtering, malware and network intrusion detection, among others. Despite the capabilities of machine learning algorithms to extract valuable information from data and produce accurate predictions, it has been shown that these algorithms are vulnerable to attacks. Data poisoning is one of the most relevant security threats against machine learning systems, where attackers can subvert the learning process by injecting malicious samples in the training data. Recent work in adversarial machine learning has shown that the so-called optimal attack strategies can successfully poison linear classifiers, degrading the performance of the system dramatically after compromising a small fraction of the training dataset. In this paper we propose a defence mechanism to mitigate the effect of these optimal poisoning attacks based on outlier detection. We show empirically that the adversarial examples generated by these attack strategies are quite different from genuine points, as no detectability constrains are considered to craft the attack. Hence, they can be detected with an appropriate pre-filtering of the training dataset.

Motivation & Objective

• Motivate the threat of data poisoning in machine learning systems and its impact on performance.
• Propose a defence that detects adversarial poisoning points via outlier detection across the training pipeline.
• Provide a computationally efficient, algorithm-agnostic mitigation strategy that does not require prior knowledge of the poisoning fraction.
• Evaluate the approach on synthetic and real datasets, including high-dimensional scenarios, and compare against optimal and constrained attacks.

Proposed method

• Formulate poisoning as a bilevel optimization problem where an attacker optimizes poisoning points to maximize a validation objective.
• Describe an optimal poisoning attack against Lasso-like linear classifiers using an explicit gradient-based update and implicit differentiation via the KKT conditions.
• Propose a defence that applies distance-based outlier detection on a small trusted subset to identify and filter poisoning points before retraining.
• Use a two-class outlier detector with class-specific thresholds to decide which samples to remove during re-training.
• Discuss practical considerations such as the curse of dimensionality and the need for trusted data to train the detectors.

Experimental results

Research questions

• RQ1 Can distance-based outlier detection effectively identify adversarial training examples injected into a training dataset under poisoning attacks?
• RQ2 How does the proposed pre-filtering approach affect the impact of optimal poisoning strategies on linear classifiers in various data regimes (e.g., few samples, high feature count)?

Key findings

• Outlier detection can mitigate the effect of optimal poisoning strategies on linear classifiers by pre-filtering adversarial training points.
• The defense is computationally efficient and does not require knowing the poisoning fraction in advance.
• Label-flipping type attacks are harder to detect with the proposed outlier detector than the optimal poisoning strategies.
• The approach remains effective even when the number of training points is comparable to the number of features.
• The defense is agnostic to the underlying learning algorithm and applicable to large datasets.