[Paper Review] Secure Weighted Aggregation for Federated Learning

Jiale Guo, Ziyao Liu+4 more|arXiv (Cornell University)|Oct 17, 2020

Privacy-Preserving Technologies in DataComputer Science20 references17 citations

TL;DR

This paper proposes a privacy-enhanced federated learning scheme that enables secure weighted aggregation using homomorphic encryption and zero-knowledge proofs to handle data disparity and resist fraudulent client messages. The scheme achieves secure, weighted model aggregation with only 1.2× runtime and 1.3× communication overhead compared to baseline methods, ensuring both data privacy and integrity.

ABSTRACT

The pervasive adoption of Internet-connected digital services has led to a growing concern in the personal data privacy of their customers. On the other hand, machine learning (ML) techniques have been widely adopted by digital service providers to improve operational productivity and customer satisfaction. ML inevitably accesses and processes users' personal data, which could potentially breach the relevant privacy protection regulations if not performed carefully. The situation is exacerbated by the cloud-based implementation of digital services when user data are captured and stored in distributed locations, hence aggregation of the user data for ML could be a serious breach of privacy regulations. In this backdrop, Federated Learning (FL) is an emerging area that allows ML on distributed data without the data leaving their stored location. However, depending on the nature of the digital services, data captured at different locations may carry different significance to the business operation, hence a weighted aggregation will be highly desirable for enhancing the quality of the FL-learned model. Furthermore, to prevent leakage of user data from the aggregated gradients, cryptographic mechanisms are needed to allow secure aggregation of FL. In this paper, we propose a privacy-enhanced FL scheme for supporting secure weighted aggregation. Besides, by devising a verification protocol based on Zero-Knowledge Proof (ZKP), the proposed scheme is capable of guarding against fraudulent messages from FL participants. Experimental results show that our scheme is practical and secure. Compared to existing FL approaches, our scheme achieves secure weighted aggregation with an additional security guarantee against fraudulent messages with an affordable 1.2 times runtime overheads and 1.3 times communication costs.

Motivation & Objective

• Address the challenge of data disparity in federated learning, where local datasets vary in size and quality, requiring weighted aggregation to improve global model accuracy.
• Ensure strong privacy guarantees by preventing leakage of local data through secure aggregation using homomorphic encryption.
• Mitigate the risk of malicious or fraudulent clients sending manipulated gradients to manipulate model weights.
• Provide a dropout-resilient aggregation mechanism to maintain system robustness when some clients fail to participate.
• Integrate zero-knowledge proofs to verify the correctness of client messages without revealing sensitive information.

Proposed method

• Leverage the Paillier homomorphic encryption scheme to enable secure aggregation of client gradients without exposing raw data.
• Use a weighted aggregation mechanism where client contributions are scaled by data size and label quality metrics to reflect dataset significance.
• Design a zero-knowledge proof-based verification protocol to validate that each client's uploaded gradient is correctly computed and consistent with their claimed data size and quality.
• Integrate a dropout-resilient secure aggregation framework to maintain system availability and correctness when some clients fail to respond.
• Apply threshold secret sharing and secure multi-party computation principles to distribute trust and prevent single-point compromise.
• Combine the verification protocol with the aggregation pipeline to detect and reject fraudulent messages before model update.

Experimental results

Research questions

• RQ1 How can federated learning systems securely aggregate model updates while accounting for data disparity across clients?
• RQ2 What cryptographic mechanisms can ensure privacy-preserving aggregation without exposing client data or gradients?
• RQ3 How can the system detect and prevent malicious clients from submitting fraudulent gradients to manipulate the global model?
• RQ4 What is the performance overhead of integrating zero-knowledge proofs into secure federated learning with weighted aggregation?
• RQ5 Can the proposed scheme maintain robustness and correctness in the presence of client dropouts?

Key findings

• The proposed scheme achieves secure weighted aggregation with only 1.2 times the runtime and 1.3 times the communication cost of baseline methods, demonstrating practicality.
• The zero-knowledge proof-based verification protocol successfully detects and rejects fraudulent messages without revealing sensitive client data.
• The system remains resilient to client dropouts, maintaining secure aggregation even when up to 30% of clients fail to respond in either phase.
• The integration of homomorphic encryption and zero-knowledge proofs ensures end-to-end privacy and integrity, preventing data leakage and model poisoning.
• Experimental results show that the scheme maintains high model accuracy while providing strong security guarantees against data leakage and adversarial manipulation.
• The scheme outperforms existing approaches in handling data disparity and adversarial behavior, offering a balanced trade-off between security, accuracy, and efficiency.