[论文解读] Contamination Attacks and Mitigation in Multi-Party Machine Learning

Jamie Hayes, Olga Ohrimenko|arXiv (Cornell University)|Jan 8, 2019

Adversarial Robustness in Machine Learning被引用 36

一句话总结

本文在集中式多方机器学习中定义了污染攻击，并表明对抗性训练可以防御这些攻击，同时在保留模型效用的同时减少参与方成员身份推断风险。

ABSTRACT

Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time. Distinct parties may want to combine their local data to gain the benefits of a model trained on a large corpus of data. We consider such a case: parties get access to the model trained on their joint data but do not see each others individual datasets. We show that one needs to be careful when using this multi-party model since a potentially malicious party can taint the model by providing contaminated data. We then show how adversarial training can defend against such attacks by preventing the model from learning trends specific to individual parties data, thereby also guaranteeing party-level membership privacy.

研究动机与目标

• Understand how malicious parties can influence multi-party models through contaminated data.
• Demonstrate contamination attacks using real datasets with different data modalities.
• Propose and evaluate adversarial training as a defense that preserves utility while reducing leak of party information.
• Show that adversarial training also mitigates party membership inference risks.

提出的方法

• Define contamination attack methodology and attacker budget in a centralized multi-party ML setting.
• Demonstrate attacks on three datasets (Adult, Credit Card, News20) with both categorical and text data.
• Apply neural network and CNN-based models across datasets and measure contamination and validation accuracy.
• Propose adversarial training where a secondary model attempts to predict party identity to pivot away from party-specific information.
• Evaluate two adversarial training variants (f' and f'') and analyze effects on contamination and privacy metrics.
• Provide theoretical justification that adversarial training yields party-level privacy and mitigates contamination without sacrificing performance.

实验结果

研究问题

• RQ1 How much can a malicious party influence what is learned during training in a centralized multi-party ML setting?
• RQ2 Can contamination attacks be detected or mitigated without knowledge of which attributes/labels are targeted?
• RQ3 Does adversarial training reduce contamination while maintaining model accuracy and providing party-level privacy?
• RQ4 Can adversarial training also mitigate party-level membership inference attacks?

主要发现

• Contamination attacks can create artificial correlations between contaminated attributes and chosen labels, affecting the multi-party model.
• Adversarial training mitigates contamination attacks and can preserve or improve validation accuracy relative to non-adversarial training.
• Adversarial training reduces contamination accuracy to near local-model levels even when attackers control multiple parties.
• The approach provides party-level privacy by making model outputs independent of the party identifier.
• Adversarial training also substantially reduces party-level membership inference risk.
• Data from different distributions causes only minor drops in validation accuracy under adversarial training.