[Paper Review] Backdoor attacks and defenses in feature-partitioned collaborative learning

Yang Liu, Zhihao Yi+1 more|arXiv (Cornell University)|Jul 7, 2020

Privacy-Preserving Technologies in DataComputer Science19 references36 citations

TL;DR

The paper demonstrates that backdoor attacks can be injected by passive parties in feature-partitioned collaborative learning and proposes defense strategies, including a trainable active party, noise, and gradient sparsification.

ABSTRACT

Since there are multiple parties in collaborative learning, malicious parties might manipulate the learning process for their own purposes through backdoor attacks. However, most of existing works only consider the federated learning scenario where data are partitioned by samples. The feature-partitioned learning can be another important scenario since in many real world applications, features are often distributed across different parties. Attacks and defenses in such scenario are especially challenging when the attackers have no labels and the defenders are not able to access the data and model parameters of other participants. In this paper, we show that even parties with no access to labels can successfully inject backdoor attacks, achieving high accuracy on both main and backdoor tasks. Next, we introduce several defense techniques, demonstrating that the backdoor can be successfully blocked by a combination of these techniques without hurting main task accuracy. To the best of our knowledge, this is the first systematical study to deal with backdoor attacks in the feature-partitioned collaborative learning framework.

Motivation & Objective

• Motivate the study of backdoor attacks in feature-partitioned collaborative learning where features are distributed across parties.
• Show that passive parties without labels can inject backdoors by manipulating exchanged messages.
• Propose defense techniques and evaluate their effectiveness without harming the main task performance.

Proposed method

• Formalize a feature-partitioned collaborative learning framework with an active party (labels) and passive parties (features).
• Model backdoor attacks as gradient/message poisoning and gradient-replacement strategies across communication rounds.
• Propose defenses including adding trainable layers at the active party, differential privacy (noise), and gradient sparsification.
• Provide algorithmic details for gradient poisoning and activation blurring to realize backdoors.
• Conduct empirical evaluation on MNIST and NUS-WIDE datasets to assess attack success and defense effectiveness.

Experimental results

Research questions

• RQ1 Can passive parties in feature-partitioned collaborative learning perform backdoor attacks without access to labels?
• RQ2 How can backdoors be injected via exchanged gradients, and how can they be detected or blocked?
• RQ3 Do defenses like a trainable active party, differential privacy, and gradient sparsification effectively mitigate backdoors without harming the main task?

Key findings

• Backdoor attacks can succeed in feature-partitioned learning by manipulating inter-party messages, achieving high accuracy on both main and backdoor tasks.
• Activating defenses with a trainable active party can reduce label leakage and improve model performance; however, effectiveness varies by dataset.
• Differential privacy (noise addition) can mitigate backdoors but may degrade main-task accuracy at higher noise levels.
• Gradient sparsification can mitigate backdoors and preserve main-task accuracy, especially when combined with other defenses.
• Attacks are feasible even when the attacker controls passive parties with no labels, and the backdoor can persist across iterations.