[Paper Review] Contamination Attacks and Mitigation in Multi-Party Machine Learning

Jamie Hayes, Olga Ohrimenko|arXiv (Cornell University)|Jan 8, 2019

Adversarial Robustness in Machine Learning36 citations

TL;DR

The paper defines contamination attacks in centralized multi-party ML and shows adversarial training can defend against them while preserving model utility and also reducing party membership inference risks.

ABSTRACT

Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time. Distinct parties may want to combine their local data to gain the benefits of a model trained on a large corpus of data. We consider such a case: parties get access to the model trained on their joint data but do not see each others individual datasets. We show that one needs to be careful when using this multi-party model since a potentially malicious party can taint the model by providing contaminated data. We then show how adversarial training can defend against such attacks by preventing the model from learning trends specific to individual parties data, thereby also guaranteeing party-level membership privacy.

Motivation & Objective

• Understand how malicious parties can influence multi-party models through contaminated data.
• Demonstrate contamination attacks using real datasets with different data modalities.
• Propose and evaluate adversarial training as a defense that preserves utility while reducing leak of party information.
• Show that adversarial training also mitigates party membership inference risks.

Proposed method

• Define contamination attack methodology and attacker budget in a centralized multi-party ML setting.
• Demonstrate attacks on three datasets (Adult, Credit Card, News20) with both categorical and text data.
• Apply neural network and CNN-based models across datasets and measure contamination and validation accuracy.
• Propose adversarial training where a secondary model attempts to predict party identity to pivot away from party-specific information.
• Evaluate two adversarial training variants (f' and f'') and analyze effects on contamination and privacy metrics.
• Provide theoretical justification that adversarial training yields party-level privacy and mitigates contamination without sacrificing performance.

Experimental results

Research questions

• RQ1 How much can a malicious party influence what is learned during training in a centralized multi-party ML setting?
• RQ2 Can contamination attacks be detected or mitigated without knowledge of which attributes/labels are targeted?
• RQ3 Does adversarial training reduce contamination while maintaining model accuracy and providing party-level privacy?
• RQ4 Can adversarial training also mitigate party-level membership inference attacks?

Key findings

• Contamination attacks can create artificial correlations between contaminated attributes and chosen labels, affecting the multi-party model.
• Adversarial training mitigates contamination attacks and can preserve or improve validation accuracy relative to non-adversarial training.
• Adversarial training reduces contamination accuracy to near local-model levels even when attackers control multiple parties.
• The approach provides party-level privacy by making model outputs independent of the party identifier.
• Adversarial training also substantially reduces party-level membership inference risk.
• Data from different distributions causes only minor drops in validation accuracy under adversarial training.