[Paper Review] An Efficient and Multi-private Key Secure Aggregation for Federated Learning

Xue Yang, Zifeng Liu+3 more|arXiv (Cornell University)|Jun 15, 2023

Privacy-Preserving Technologies in DataComputer Science3 citations

TL;DR

This paper proposes a novel, efficient, and multi-private key secure aggregation scheme for federated learning that eliminates the need for a trusted third party by enabling clients to independently select their own public and private keys. By modifying variant ElGamal encryption and using a super-increasing sequence to compress high-dimensional model parameters, the scheme achieves semantic security for both individual gradients and aggregated results, with strong robustness against client collusion and dropped clients, while significantly improving computational and communication efficiency over existing homomorphic encryption-based approaches.

ABSTRACT

With the emergence of privacy leaks in federated learning, secure aggregation protocols that mainly adopt either homomorphic encryption or threshold secret sharing have been widely developed for federated learning to protect the privacy of the local training data of each client. However, these existing protocols suffer from many shortcomings, such as the dependence on a trusted third party, the vulnerability to clients being corrupted, low efficiency, the trade-off between security and fault tolerance, etc. To solve these disadvantages, we propose an efficient and multi-private key secure aggregation scheme for federated learning. Specifically, we skillfully modify the variant ElGamal encryption technique to achieve homomorphic addition operation, which has two important advantages: 1) The server and each client can freely select public and private keys without introducing a trust third party and 2) Compared to the variant ElGamal encryption, the plaintext space is relatively large, which is more suitable for the deep model. Besides, for the high dimensional deep model parameter, we introduce a super-increasing sequence to compress multi-dimensional data into 1-D, which can greatly reduce encryption and decryption times as well as communication for ciphertext transmission. Detailed security analyses show that our proposed scheme achieves the semantic security of both individual local gradients and the aggregated result while achieving optimal robustness in tolerating both client collusion and dropped clients. Extensive simulations demonstrate that the accuracy of our scheme is almost the same as the non-private approach, while the efficiency of our scheme is much better than the state-of-the-art homomorphic encryption-based secure aggregation schemes. More importantly, the efficiency advantages of our scheme will become increasingly prominent as the number of model parameters increases.

Motivation & Objective

• To address the limitations of existing secure aggregation protocols in federated learning, including reliance on trusted third parties, vulnerability to client compromise, and high computational and communication overhead.
• To eliminate the need for a trusted third party in key generation while maintaining strong security guarantees for both individual gradients and aggregated results.
• To improve efficiency for deep learning models by reducing encryption/decryption operations and ciphertext transmission through dimensionality compression using a super-increasing sequence.
• To achieve optimal robustness against both client collusion and dropped clients without sacrificing fault tolerance or security.
• To ensure the privacy of the aggregated result, not just individual gradients, by preventing unauthorized access to transmitted data.

Proposed method

• The scheme modifies variant ElGamal encryption to support additive homomorphic operations under independently chosen public and private keys, enabling secure aggregation without a trusted third party.
• Each client uses its own public and private key pair to encrypt its local gradient, ensuring that compromise of one client does not break the entire system.
• A super-increasing sequence is applied to compress multi-dimensional model parameters into a 1D vector, reducing the number of encryption and decryption operations and minimizing ciphertext transmission.
• The protocol ensures that only the server can decrypt the aggregated result, while individual gradients remain semantically secure.
• Security is achieved through careful design of a secure interactive protocol that maintains homomorphic properties under multi-private key settings.
• The scheme supports optimal fault tolerance, allowing up to n−t dropped or colluding clients, where t is the threshold for reconstruction.

Experimental results

Research questions

• RQ1 Can a secure aggregation scheme for federated learning be designed without relying on a trusted third party for key generation?
• RQ2 How can homomorphic encryption be adapted to support additive operations when clients use independently selected public and private keys?
• RQ3 What techniques can effectively compress high-dimensional model parameters to reduce encryption/decryption and communication overhead?
• RQ4 How can the scheme maintain semantic security for both individual gradients and the aggregated result under client collusion and dropped client scenarios?
• RQ5 Can the scheme achieve better efficiency than existing homomorphic encryption-based secure aggregation protocols, especially as model size increases?

Key findings

• The proposed scheme achieves semantic security for both individual local gradients and the aggregated result, even under client collusion and dropped client conditions.
• The scheme eliminates the need for a trusted third party, enhancing system security and decentralization.
• By using a modified variant of ElGamal encryption, the plaintext space is significantly expanded, making it more suitable for deep learning models with large parameter dimensions.
• The use of a super-increasing sequence reduces the number of encryption and decryption operations and minimizes ciphertext transmission, leading to substantial efficiency gains.
• Performance evaluations show that the scheme's computational and communication efficiency outperforms state-of-the-art homomorphic encryption-based schemes, with advantages becoming more pronounced as model size increases.
• The accuracy of the proposed scheme is nearly identical to non-private federated learning, confirming minimal impact on model performance.