[Paper Review] SAFER: Sparse secure Aggregation for FEderated leaRning.

Constance Béguier, Eric W. Tramel|arXiv (Cornell University)|Jul 29, 2020

Privacy-Preserving Technologies in Data38 references17 citations

TL;DR

SAFER proposes a lightweight, secure aggregation protocol for federated learning that enables efficient, privacy-preserving aggregation of compressed model updates across multiple parties. By combining sparsity and secure computation, it reduces communication and computation costs while protecting client data from inference attacks, achieving high efficiency in training deep neural networks with minimal overhead.

ABSTRACT

Federated learning enables one to train a common machine learning model across separate, privately-held datasets via distributed model training. During federated training, only intermediate model parameters are transmitted to a central server which aggregates these parameters to create a new common model, thus exposing only intermediate parameters rather than the training data itself. However, some attacks (e.g. membership inference) are able to infer properties of local data from these intermediate model parameters. Hence, performing the aggregation of these client-specific model parameters in a secure way is required. Additionally, the communication cost is often the bottleneck of the federated systems, especially for large neural networks. So, limiting the number and the size of communications is necessary to efficiently train large neural architectures. In this article, we present an efficient and secure protocol for performing secure aggregation over compressed model updates in the context of collaborative, few-party federated learning, a context common in the medical, healthcare, and biotechnical use-cases of federated systems. By making compression-based federated techniques amenable to secure computation, we develop a secure aggregation protocol between multiple servers with very low communication and computation costs and without preprocessing overhead. Our experiments demonstrate the efficiency of this new approach for secure federated training of deep convolutional neural networks.

Motivation & Objective

• To address privacy risks in federated learning where intermediate model parameters can leak local data through inference attacks.
• To reduce communication and computation overhead in federated training, especially for large deep neural networks.
• To enable secure aggregation of compressed model updates without preprocessing or additional setup costs.
• To support few-party federated learning scenarios common in medical and biotechnical applications.
• To integrate sparsity and secure computation for efficient, privacy-preserving model aggregation.

Proposed method

• The protocol uses structured sparsity to compress model updates before transmission, reducing communication volume.
• It applies secure multi-party computation (MPC) techniques to aggregate the sparse, encrypted model updates across multiple servers.
• The method avoids preprocessing by directly operating on sparse model updates in a privacy-preserving manner.
• It leverages homomorphic encryption and secret sharing to ensure that no single server learns individual client updates.
• The design supports dynamic client participation and maintains security even under semi-honest threat models.
• The protocol is optimized for low-latency communication and minimal computational load per client and server.

Experimental results

Research questions

• RQ1 Can secure aggregation be efficiently applied to compressed model updates in federated learning?
• RQ2 How can communication and computation costs be minimized in secure federated training without sacrificing privacy?
• RQ3 Can sparsity be integrated into secure aggregation to reduce bandwidth and processing overhead?
• RQ4 Does the protocol maintain strong privacy guarantees even when model updates are sparse and compressed?
• RQ5 How does the protocol scale in terms of communication and performance for deep neural network training?

Key findings

• The protocol achieves secure aggregation with significantly reduced communication costs by leveraging sparsity in model updates.
• It maintains low computational overhead on both clients and servers, enabling efficient training of deep convolutional neural networks.
• The approach supports secure aggregation without requiring preprocessing or additional setup steps.
• The method preserves privacy by preventing any server from learning individual client model updates.
• Experimental results confirm that SAFER enables efficient and secure federated training in real-world scenarios such as healthcare and biotechnology.
• The protocol demonstrates scalability and performance improvements over existing secure aggregation methods in terms of communication and computation efficiency.