[Paper Review] Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

Matthew Jagielski, Alina Oprea+4 more|arXiv (Cornell University)|Apr 1, 2018

Adversarial Robustness in Machine Learning38 references113 citations

TL;DR

This paper pioneers poisoning attacks against linear regression, proposes an optimization-based and a fast statistical attack, and introduces a robust defense TRIM with convergence guarantees and lower MSE impact.

ABSTRACT

As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms. In this paper, we perform the first systematic study of poisoning attacks and their countermeasures for linear regression models. In poisoning attacks, attackers deliberately influence the training data to manipulate the results of a predictive model. We propose a theoretically-grounded optimization framework specifically designed for linear regression and demonstrate its effectiveness on a range of datasets and models. We also introduce a fast statistical attack that requires limited knowledge of the training process. Finally, we design a new principled defense method that is highly resilient against all poisoning attacks. We provide formal guarantees about its convergence and an upper bound on the effect of poisoning attacks when the defense is deployed. We evaluate extensively our attacks and defenses on three realistic datasets from health care, loan assessment, and real estate domains.

Motivation & Objective

• Introduce and formalize poisoning attacks for linear regression under white-box and black-box settings.
• Develop an optimization-based poisoning framework tailored to regression and extend it to optimize both features and responses.
• Propose a fast statistical poisoning attack requiring limited training-process knowledge.
• Design a principled defense (TRIM) with convergence guarantees and provable bounds on poisoning impact.
• Evaluate attacks and defenses across multiple regression models and real-world datasets.

Proposed method

• Adapt a baseline poisoning attack from classification to regression and extend it with joint optimization of poisoning features and responses.
• Formulate poisoning as a bilevel optimization problem with an inner training objective and an outer poisoning objective.
• Compute gradients through the KKT conditions of the inner convex optimization to update poisoning points.
• Introduce a fast statistical attack (StatP) that samples from the training distribution and uses boundary rounding to maximize loss, requiring only black-box access.
• Propose TRIM, a robust regression method using trimmed loss to iteratively prune high-residual points and learn a resilient model.
• Provide convergence guarantees for TRIM and an upper bound on mean squared error under poisoning.

Experimental results

Research questions

• RQ1 How do poisoning attacks affect linear regression models under different regularization schemes (OLS, Ridge, LASSO, Elastic Net)?
• RQ2 Can poisoning be effectively performed with limited attacker knowledge, including black-box scenarios?
• RQ3 What defense mechanisms can provide robustness against a broad class of poisoning attacks in regression, and what are their theoretical guarantees?
• RQ4 How do optimization-based and statistical poisoning approaches compare in effectiveness and efficiency?
• RQ5 What are the empirical impacts of poisoning on real-world regression tasks across healthcare, lending, and real estate domains?

Key findings

• Optimization-based poisoning attacks significantly increase testing MSE across multiple regression models and datasets compared with prior work.
• The attacks can amplify MSE by large factors (e.g., up to 6.83× compared to a baseline and 155.7× versus unpoisoned models) on reported cases.
• The proposed TRIM defense outperforms robust statistics methods (Huber, RANSAC) and other adversarial defenses, with a median MSE increase of only 6.1% across attacks, models, and datasets.
• TRIM provides convergence guarantees and an upper bound on poisoning impact, demonstrating strong robustness against a wide class of attacks.
• The StatP attack, while faster and requiring less information, remains effective and operates with black-box access by sampling from data distribution and using boundary rounding.
• Compared to the Xiao et al. baseline, the optimization-based attacks show significant improvements in poisoning efficacy.