[Paper Review] Federated Learning with Differential Privacy: Algorithms and Performance Analysis

Kang Wei, Jun Li+7 more|arXiv (Cornell University)|Nov 1, 2019

Privacy-Preserving Technologies in Data35 references88 citations

TL;DR

Proposes NbAFL, a differential privacy-based federated learning framework that adds noise before aggregation, and provides theoretical convergence bounds and a K-random scheduling variant.

ABSTRACT

In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noises are added to the parameters at the clients side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noises. Then we develop a theoretical convergence bound of the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i.e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a $K$-random scheduling strategy, where $K$ ($1

Motivation & Objective

• Motivate privacy-preserving federated learning to prevent information leakage from uploaded parameters.
• Introduce Noising before Aggregation FL (NbAFL) that satisfies DP by adding Gaussian noise at clients and server.
• Derive theoretical convergence bounds for NbAFL under DP, revealing privacy-utility tradeoffs and optimal aggregation parameters.
• Propose a K-random scheduling scheme and analyze its impact on convergence and privacy.
• Provide guidance for designing privacy-preserving FL algorithms with configurable privacy-utility tradeoffs.

Proposed method

• Formulate NbAFL where clients clip and add Gaussian noise to local updates before uploading, and the server adds noise to the aggregated global model before broadcasting.
• Prove that NbAFL satisfies (ε,δ)-DP under Gaussian mechanisms with appropriately chosen noise scales.
• Derive a convergence bound for NbAFL showing a tradeoff among convergence performance, privacy level ε, number of clients N, and aggregation rounds T.
• Extend analysis to a K-random scheduling scheme where K clients participate per aggregation and derive corresponding DP guarantees and convergence bounds.
• Provide closed-form expressions for noise scales in uplink and downlink channels and conditions under which DP is maintained.

Experimental results

Research questions

• RQ1 How can differential privacy be integrated into federated learning with noising before aggregation (NbAFL) while preserving model utility?
• RQ2 What are the convergence properties of NbAFL under DP, and how do privacy levels, client count, and aggregation rounds interact?
• RQ3 How does a K-random scheduling of clients affect DP guarantees and convergence performance compared to full-client aggregation?
• RQ4 What are optimal settings (e.g., aggregation rounds, K) to balance privacy protection and convergence accuracy?

Key findings

• NbAFL can satisfy (ε,δ)-DP by adapting Gaussian noise variances at the client side and server side.
• Theoretical convergence bounds show a tradeoff: better convergence corresponds to lower privacy protection (larger ε).
• Increasing the number of participating clients N improves convergence under a fixed privacy level.
• There exists an optimal number of maximum aggregation rounds T for given privacy level and N.
• With K-random scheduling, a corresponding DP guarantee and convergence bound hold, and there exists an optimal K for best convergence at a fixed privacy level.
• Simulations validate the theoretical bounds and demonstrate alignment with the proposed DP-driven design.