[论文解读] LightSecAgg: Rethinking Secure Aggregation in Federated Learning

Chien-Sheng Yang, Jinhyun So+4 more|arXiv (Cornell University)|Sep 29, 2021

Privacy-Preserving Technologies in Data参考文献 32被引用 14

一句话总结

LightSecAgg 提出了一种用于联邦学习的新型安全聚合协议，通过允许服务器直接从活跃用户的编码掩码重建聚合掩码，实现了高效且对丢包具有鲁棒性的模型聚合，相比基线方法训练速度最高提升12.7倍，同时保持了强大的隐私保护和鲁棒性保证。

ABSTRACT

Secure model aggregation is a key component of federated learning (FL) that aims at protecting the privacy of each user's individual model, while allowing their global aggregation. It can be applied to any aggregation-based approaches, including algorithms for training a global model, as well as personalized FL frameworks. Model aggregation needs to also be resilient to likely user dropouts in FL system, making its design substantially more complex. State-of-the-art secure aggregation protocols essentially rely on secret sharing of the random-seeds that are used for mask generations at the users, in order to enable the reconstruction and cancellation of those belonging to dropped The complexity of such approaches, however, grows substantially with the number of dropped We propose a new approach, named LightSecAgg, to overcome this bottleneck by turning the focus from random-seed reconstruction of the dropped to one-shot aggregate-mask reconstruction of the active users. More specifically, in LightSecAgg each user protects its local model by generating a single random mask. This mask is then encoded and shared to other users, in such a way that the aggregate-mask of any sufficiently large set of active users can be reconstructed directly at the server via encoded masks. We show that LightSecAgg achieves the same privacy and dropout-resiliency guarantees as the state-of-the-art protocols, while significantly reducing the overhead for resiliency to dropped Furthermore, our system optimization helps to hide the runtime cost of offline processing by parallelizing it with model training. We evaluate LightSecAgg via extensive experiments for training diverse models on various datasets in a realistic FL system, and demonstrate that LightSecAgg significantly reduces the total training time, achieving a performance gain of up to $12.7 imes$ over baselines.

研究动机与目标

• 解决现有安全聚合协议在用户丢包时计算和通信开销高的问题，避免其可扩展性差。
• 在训练过程中用户发生丢包的情况下，仍能维持强大的隐私保护和丢包鲁棒性保证。
• 通过将重点从重建已丢弃用户的随机种子，转向直接从活跃用户的掩码重建聚合掩码，降低安全聚合的运行时成本。
• 通过将离线处理与模型训练并行化，隐藏其延迟，从而优化系统性能。
• 实现在频繁用户丢包的现实动态联邦学习环境中，安全聚合的高效部署。

提出的方法

• 每个用户在将本地模型发送给服务器之前，生成一个单一的随机掩码以保护其模型。
• 用户使用秘密共享方案对掩码进行编码，使得任意足够大的活跃用户子集都能在服务器端联合重建聚合掩码。
• 服务器直接从活跃用户的编码掩码中重建聚合掩码，无需重建已丢弃用户的随机种子。
• 该协议利用编码的线性代数特性，在威胁模型下确保正确性和隐私性。
• 将离线编码计算与模型训练并行化，以隐藏其运行时成本。
• 系统设计兼容全局和个性化联邦学习框架。

实验结果

研究问题

• RQ1 是否可以通过避免重建已丢弃用户的随机种子，使联邦学习中的安全聚合更加高效？
• RQ2 如何仅从活跃用户的编码掩码中安全且高效地实现聚合掩码的重建？
• RQ3 系统级优化（如将离线处理与训练并行化）在多大程度上能减少安全联邦学习的端到端训练延迟？
• RQ4 所提出的方法是否能维持与最先进协议相当的强大隐私保护和丢包鲁棒性保证？
• RQ5 在高用户丢包率的真实联邦学习环境中，该方法能实现多大的性能提升？

主要发现

• 在真实联邦学习环境中，LightSecAgg 相比基线安全聚合协议，将总训练时间最多减少了12.7倍。
• 该协议维持了与最先进方法相当的隐私保护和丢包鲁棒性，即使部分用户丢包，也不会暴露任何用户的模型。
• 通过将重点从随机种子重建转向直接聚合掩码重建，LightSecAgg 显著降低了与用户丢包处理相关的计算和通信开销。
• 系统级优化（包括将离线编码与模型训练并行化）有效隐藏了预处理延迟，提升了实际可用性。
• 该方法兼容多种模型和数据集，在不同联邦学习工作负载中均表现出一致的性能提升。
• 随着丢弃用户数量的增加，该协议仍能高效扩展，因为其复杂度不随丢包数量增长。