[Paper Review] Practical and Bilateral Privacy-preserving Federated Learning

Yan Feng, Xue Yang+3 more|arXiv (Cornell University)|Feb 23, 2020

Privacy-Preserving Technologies in Data25 references11 citations

TL;DR

This paper proposes the first bilateral privacy-preserving federated learning scheme that protects both clients' raw data and model iterates/parameters from leakage. It uses an efficient masking technique to encrypt the global model so clients train on noisy versions while only the server can reconstruct the exact updated model, achieving strong privacy with minimal communication overhead and comparable accuracy to standard federated learning.

ABSTRACT

Federated learning, as an emerging distributed training model of neural networks without collecting raw data, has attracted widespread attention. However, almost all existing researches of federated learning only consider protecting the privacy of clients, but not preventing model iterates and final model parameters from leaking to untrusted clients and external attackers. In this paper, we present the first bilateral privacy-preserving federated learning scheme, which protects not only the raw training data of clients, but also model iterates during the training phase as well as final model parameters. Specifically, we present an efficient privacy-preserving technique to mask or encrypt the global model, which not only allows clients to train over the noisy global model, but also ensures only the server can obtain the exact updated model. Detailed security analysis shows that clients can access neither model iterates nor the final global model; meanwhile, the server cannot obtain raw training data of clients from additional information used for recovering the exact updated model. Finally, extensive experiments demonstrate the proposed scheme has comparable model accuracy with traditional federated learning without bringing much extra communication overhead.

Motivation & Objective

• Address the gap in federated learning where only client data privacy is protected, but model updates and final parameters remain exposed.
• Prevent untrusted clients and external adversaries from inferring sensitive model information during training.
• Ensure the server cannot access raw client data even when using auxiliary information to recover model updates.
• Design a practical scheme that maintains model accuracy while adding minimal communication and computational overhead.

Proposed method

• Propose a novel privacy-preserving technique to mask or encrypt the global model during federated training.
• Allow clients to train on a noisy version of the global model, preventing them from observing exact model parameters.
• Ensure only the server can reconstruct the exact updated model using auxiliary information.
• Use cryptographic masking that enables model updates to be computed securely without exposing intermediate or final model states.
• Integrate the masking mechanism into the standard federated averaging framework with minimal modifications.
• Maintain compatibility with existing federated learning systems while enhancing privacy guarantees.

Experimental results

Research questions

• RQ1 Can a federated learning system simultaneously protect client data and model parameters from both clients and external adversaries?
• RQ2 How can model iterates and final model weights be hidden from untrusted clients without compromising training accuracy?
• RQ3 What is the communication and computational overhead of protecting model information in addition to data?
• RQ4 Can the server reconstruct the exact model updates without learning raw client data from auxiliary information?

Key findings

• The proposed scheme achieves comparable model accuracy to standard federated learning, with no significant degradation in performance.
• The communication overhead introduced by the privacy-preserving mechanism remains minimal and comparable to baseline federated learning.
• Clients cannot access model iterates or the final global model, as they only receive masked versions of the model.
• The server cannot infer raw training data from auxiliary information used to recover the exact model updates.
• Security analysis confirms that both client data and model parameters are protected against leakage to untrusted parties.