[论文解读] Federated Robustness Propagation: Sharing Adversarial Robustness in Federated Learning.

Junyuan Hong, Haotao Wang+2 more|arXiv (Cornell University)|Jun 18, 2021

Adversarial Robustness in Machine Learning参考文献 52被引用 6

一句话总结

本文提出联邦鲁棒性传播（Federated Robustness Propagation），一种将高资源联邦学习（FL）用户通过对抗训练获得的对抗鲁棒性，传递给数据和计算资源有限的低资源用户的方法。通过在联邦学习过程中战略性地共享批量归一化统计信息，该方法使即使不具备对抗训练能力的用户也能实现强大的鲁棒性，以极低的代价显著提升联邦学习系统的整体鲁棒性。

ABSTRACT

Federated learning (FL) emerges as a popular distributed learning schema that learns a model from a set of participating users without requiring raw data to be shared. One major challenge of FL comes from heterogeneity in users, which may have distributionally different (or non-iid) data and varying computation resources. Just like in centralized learning, FL users also desire model robustness against malicious attackers at test time. Whereas adversarial training (AT) provides a sound solution for centralized learning, extending its usage for FL users has imposed significant challenges, as many users may have very limited training data as well as tight computational budgets, to afford the data-hungry and costly AT. In this paper, we study a novel learning setting that propagates adversarial robustness from high-resource users that can afford AT, to those low-resource users that cannot afford it, during the FL process. We show that existing FL techniques cannot effectively propagate adversarial robustness among non-iid users, and propose a simple yet effective propagation approach that transfers robustness through carefully designed batch-normalization statistics. We demonstrate the rationality and effectiveness of our method through extensive experiments. Especially, the proposed method is shown to grant FL remarkable robustness even when only a small portion of users afford AT during learning. Codes will be published upon acceptance.

研究动机与目标

• 解决用户数据分布异构且计算资源有限时，联邦学习中的对抗鲁棒性挑战。
• 使无法因数据和计算资源限制而执行对抗训练的低资源用户，能够从具备对抗训练能力的高资源用户处受益于鲁棒性传播。
• 克服现有联邦学习方法在非独立同分布（non-i.i.i.d.）和资源受限用户之间传播鲁棒性的局限性。
• 设计一种轻量、高效的机制，在无需低资源客户端额外数据或训练的情况下实现鲁棒性传递。

提出的方法

• 该方法利用执行对抗训练的高资源用户所获得的批量归一化统计信息，将鲁棒性传播至低资源客户端。
• 鲁棒性被编码在批量归一化层的运行统计量中，这些统计量在联邦学习聚合过程中被共享。
• 该方法无需在低资源客户端进行重训练或引入额外数据，而是依赖于模型聚合过程中的统计量迁移。
• 通过精心设计的归一化统计量，实现鲁棒性传播，同时保留对抗泛化特性。
• 该方法与标准联邦学习框架兼容，且计算开销极低。

实验结果

研究问题

• RQ1 在联邦学习中，能否有效将对抗鲁棒性从高资源用户传播至低资源用户？
• RQ2 如何在不需低资源客户端执行对抗训练的情况下实现鲁棒性传递？
• RQ3 所提出的方法在非独立同分布（non-i.i.i.d.）数据分布和客户端资源受限条件下是否仍能保持鲁棒性增益？
• RQ4 共享批量归一化统计信息对整体联邦学习鲁棒性有何影响？

主要发现

• 所提方法显著提升了联邦学习系统整体的对抗鲁棒性，即使仅少数用户执行对抗训练亦可实现。
• 得益于批量归一化统计信息的传播，低资源用户在未执行对抗训练的情况下也实现了强大的鲁棒性。
• 该方法在非独立同分布（non-i.i.i.d.）数据分布下仍保持高模型准确率，在鲁棒性方面优于标准联邦学习基线方法。
• 该方法计算开销极低，适用于资源受限客户端的真实联邦学习部署。