[論文レビュー] LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learning

Jinhyun So, Chaoyang He+6 more|arXiv (Cornell University)|Sep 29, 2021

Privacy-Preserving Technologies in Data被引用数 59

ひとこと要約

LightSecAgg は、連合学習のワンショット集約マスクセキュアアグリゲーションプロトコルを提案し、サーバー作業を削減し、非同期 FL をサポートしつつ、プライバシーとドロップアウト耐性を維持します。

ABSTRACT

Secure model aggregation is a key component of federated learning (FL) that aims at protecting the privacy of each user's individual model while allowing for their global aggregation. It can be applied to any aggregation-based FL approach for training a global or personalized model. Model aggregation needs to also be resilient against likely user dropouts in FL systems, making its design substantially more complex. State-of-the-art secure aggregation protocols rely on secret sharing of the random-seeds used for mask generations at the users to enable the reconstruction and cancellation of those belonging to the dropped users. The complexity of such approaches, however, grows substantially with the number of dropped users. We propose a new approach, named LightSecAgg, to overcome this bottleneck by changing the design from "random-seed reconstruction of the dropped users" to "one-shot aggregate-mask reconstruction of the active users via mask encoding/decoding". We show that LightSecAgg achieves the same privacy and dropout-resiliency guarantees as the state-of-the-art protocols while significantly reducing the overhead for resiliency against dropped users. We also demonstrate that, unlike existing schemes, LightSecAgg can be applied to secure aggregation in the asynchronous FL setting. Furthermore, we provide a modular system design and optimized on-device parallelization for scalable implementation, by enabling computational overlapping between model training and on-device encoding, as well as improving the speed of concurrent receiving and sending of chunked masks. We evaluate LightSecAgg via extensive experiments for training diverse models on various datasets in a realistic FL system with large number of users and demonstrate that LightSecAgg significantly reduces the total training time.

研究の動機と目的

• Motivate secure, dropout-resilient aggregation in federated learning to protect user models.
• Develop a secure aggregation protocol with lower server overhead than prior schemes.
• Enable secure aggregation in asynchronous FL settings without trusted execution environments.
• Provide a modular, open-source system design with on-device parallelization for scalable performance.

提案手法

• Introduce LightSecAgg that encodes local masks with an MDS-based scheme for one-shot aggregate-mask reconstruction.
• Three-phase protocol: offline encoding/sharing of masks, masking and uploading local models, and one-shot aggregate-model recovery.
• Three design parameters: privacy T, dropout D, and target surviving users U with N−D≥U> T.
• Extend LightSecAgg to asynchronous FL with bounded staleness by timestamped encoded masks and flexible grouping.
• Incorporate system-level optimizations and on-device parallelization to overlap training and encoding tasks.
• Prove theoretical guarantees: privacy against T colluding users and dropout resilience against D dropped users when T+D<N.

実験結果

リサーチクエスチョン

• RQ1 Can LightSecAgg provide the same privacy and dropout-resiliency guarantees as state-of-the-art protocols while reducing aggregation complexity?
• RQ2 Is LightSecAgg applicable to asynchronous federated learning and not limited to synchronous settings?
• RQ3 What is the practical performance (speed, overhead) of LightSecAgg compared with SecAgg and SecAgg+ across varied models and datasets?
• RQ4 What system-level optimizations enable scalable, real-world deployment on edge devices?

主な発見

• LightSecAgg achieves substantial speedups: 8.5×–12.7× faster than SecAgg and 2.9×–4.4× faster than SecAgg+ in realistic bandwidth settings.
• Demonstrated on diverse models (logistic regression, shallow CNNs, MobileNetV3, EfficientNet-B0) and datasets (MNIST, FEMNIST, CIFAR-10, GLD-23K).
• Supports asynchronous federated learning while preserving privacy and dropout resilience without relying on differential privacy or TEEs.
• First open-source secure aggregation-enabled FL system built on PyTorch and neural architectures with system-security co-design.
• Server-side bottleneck reduction is a primary source of the observed gains, with one-shot aggregate-mask recovery independent of the number of dropped users.