[Paper Review] On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping

Sanghyun Hong, Varun Chandrasekaran+3 more|arXiv (Cornell University)|Feb 26, 2020

Adversarial Robustness in Machine LearningComputer Science45 references70 citations

TL;DR

The paper proposes gradient shaping as an attack-agnostic defense against data poisoning by bounding gradient magnitudes and aligning gradient orientations, and evaluates DP-SGD as a practical gradient shaping tool across several models and tasks.

ABSTRACT

Machine learning algorithms are vulnerable to data poisoning attacks. Prior taxonomies that focus on specific scenarios, e.g., indiscriminate or targeted, have enabled defenses for the corresponding subset of known attacks. Yet, this introduces an inevitable arms race between adversaries and defenders. In this work, we study the feasibility of an attack-agnostic defense relying on artifacts that are common to all poisoning attacks. Specifically, we focus on a common element between all attacks: they modify gradients computed to train the model. We identify two main artifacts of gradients computed in the presence of poison: (1) their $\ell_2$ norms have significantly higher magnitudes than those of clean gradients, and (2) their orientation differs from clean gradients. Based on these observations, we propose the prerequisite for a generic poisoning defense: it must bound gradient magnitudes and minimize differences in orientation. We call this gradient shaping. As an exemplar tool to evaluate the feasibility of gradient shaping, we use differentially private stochastic gradient descent (DP-SGD), which clips and perturbs individual gradients during training to obtain privacy guarantees. We find that DP-SGD, even in configurations that do not result in meaningful privacy guarantees, increases the model's robustness to indiscriminate attacks. It also mitigates worst-case targeted attacks and increases the adversary's cost in multi-poison scenarios. The only attack we find DP-SGD to be ineffective against is a strong, yet unrealistic, indiscriminate attack. Our results suggest that, while we currently lack a generic poisoning defense, gradient shaping is a promising direction for future research.

Motivation & Objective

• Challenge the reliance on attack-specific defenses by seeking an attack-agnostic defense against data poisoning.
• Identify common gradient-level signatures of poisoned data across indiscriminate and targeted attacks.
• Propose gradient shaping as a defense principle that bounds gradient magnitudes and aligns gradient directions to mitigate poisoning.

Proposed method

• Analyze gradients during training to compare magnitudes and orientations between poison and clean samples across different poisoning scenarios.
• Craft poisons using feature collision and feature insertion to study their impact on gradients.
• Evaluate gradient-level differences using magnitude ratios and cosine similarity between poison and clean gradients.
• Instantiate gradient shaping using differentially private stochastic gradient descent (DP-SGD) as a practical tool.
• Assess DP-SGD effectiveness against indiscriminate and targeted poisoning attacks on multiple models and tasks.
• Discuss the limitations and potential of gradient shaping as a generic defense.

Experimental results

Research questions

• RQ1 Do poisoned gradients exhibit consistently higher magnitudes and differing orientations compared to clean gradients across poisoning scenarios?
• RQ2 Can gradient shaping, implemented via DP-SGD, reduce gradient-level differences and improve robustness to poisoning without relying on sanitization?
• RQ3 How effective is gradient shaping against indiscriminate versus targeted poisoning attacks on various model types and datasets?

Key findings

• Poisoned gradients generally have higher magnitudes and different orientations than clean gradients, with differences growing as poison intensity increases.
• Gradient shaping aims to reduce magnitude disparities and orientation differences to limit poisoning impact on updates.
• DP-SGD can increase robustness against indiscriminate attacks and mitigate targeted attacks even when privacy guarantees are weak.
• Gradient shaping via DP optimizers may be ineffective against strong, unrealistic indiscriminate attacks, highlighting limitations of this approach.
• Across three models and datasets, DP-SGD provided resilience improvements and increased attacker costs in multi-poison settings.
• The study identifies gradient shaping as a promising direction requiring further research to develop generic defenses.