[论文解读] RoFL: Attestable Robustness for Secure Federated Learning.

Lukas Burkhalter, Hidde Lycklama+3 more|arXiv (Cornell University)|Jul 7, 2021

Privacy-Preserving Technologies in Data参考文献 71被引用 15

一句话总结

RoFL 是一种安全的联邦学习系统，通过启用零知识证明来验证加密模型更新上的约束，从而增强对恶意客户端的鲁棒性。它通过密码学证明扩展了安全聚合协议，增加了输入检查，实现了可扩展、保护隐私的训练，并具备强大的安全保证。

ABSTRACT

Federated Learning is an emerging decentralized machine learning paradigm that allows a large number of clients to train a joint model without the need to share their private data. Participants instead only share ephemeral updates necessary to train the model. To ensure the confidentiality of the client updates, Federated Learning systems employ secure aggregation; clients encrypt their gradient updates, and only the aggregated model is revealed to the server. Achieving this level of data protection, however, presents new challenges to the robustness of Federated Learning, i.e., the ability to tolerate failures and attacks. Unfortunately, in this setting, a malicious client can now easily exert influence on the model behavior without being detected. As Federated Learning is being deployed in practice in a range of sensitive applications, its robustness is growing in importance. In this paper, we take a step towards understanding and improving the robustness of secure Federated Learning. We start this paper with a systematic study that evaluates and analyzes existing attack vectors and discusses potential defenses and assesses their effectiveness. We then present RoFL, a secure Federated Learning system that improves robustness against malicious clients through input checks on the encrypted model updates. RoFL extends Federated Learning's secure aggregation protocol to allow expressing a variety of properties and constraints on model updates using zero-knowledge proofs. To enable RoFL to scale to typical Federated Learning settings, we introduce several ML and cryptographic optimizations specific to Federated Learning. We implement and evaluate a prototype of RoFL and show that realistic ML models can be trained in a reasonable time while improving robustness.

研究动机与目标

• 为解决在安全联邦学习中，加密更新可能被操纵却无法被检测到的日益增长的风险。
• 改进依赖安全聚合以保障数据机密性的联邦学习系统的鲁棒性。
• 设计一种系统，可在不泄露敏感客户端数据的前提下验证模型更新的属性。
• 确保在具有大型模型和众多客户端的实际联邦学习环境中，验证机制的可扩展性。

提出的方法

• 扩展安全聚合协议，以支持验证加密模型更新上约束的零知识证明。
• 使用零知识证明来表达并验证梯度范数边界或稀疏性等属性，而无需暴露原始更新。
• 引入针对机器学习的优化，以降低大型神经网络的证明生成和验证的计算成本。
• 应用针对联邦学习工作负载量身定制的密码学优化，包括批量证明聚合和高效的算术电路。
• 设计一种协议，客户端提交加密更新以及正确性的零知识证明，服务器在聚合前验证这些证明。
• 通过在保持客户端数据机密性的同时实现可验证的鲁棒性，确保端到端的隐私。

实验结果

研究问题

• RQ1 如何在不损害数据隐私的前提下，防止恶意客户端操纵模型更新？
• RQ2 哪些密码学机制能够实现对加密形式下模型更新属性的高效且可扩展的验证？
• RQ3 在不降低训练性能的前提下，零知识证明在现有安全聚合协议中的集成程度如何？
• RQ4 为使基于证明的验证在大规模联邦学习和深度学习模型中具有实用性，需要哪些优化？

主要发现

• RoFL 通过零知识证明成功实现了对模型更新属性的验证，而无需暴露原始客户端梯度。
• 该系统实现了实际性能，即使存在密码学开销，也能在合理时间内训练出真实的机器学习模型。
• 将零知识证明集成到安全聚合中，通过检测并拒绝恶意更新，显著增强了系统的鲁棒性。
• 机器学习和密码学优化显著降低了证明生成和验证的成本，使 RoFL 在实际部署中具备可扩展性。
• RoFL 在保持强隐私保证的同时提供了可验证的鲁棒性，填补了安全联邦学习中的一个关键空白。