[Paper Review] Communication-Computation Efficient Secure Aggregation for Federated Learning

Beongjun Choi, Jy-yong Sohn+2 more|arXiv (Cornell University)|Dec 10, 2020

Privacy-Preserving Technologies in Data39 references48 citations

TL;DR

CCESA replaces the complete-graph secret sharing of secure aggregation with a sparse assignment graph (Erdős–Rényi), achieving similar reliability and privacy with substantially reduced communication and computation costs. The authors provide theoretical conditions and empirical results showing up to 60-70% resource savings and strong privacy under realistic attacks.

ABSTRACT

Federated learning has been spotlighted as a way to train neural networks using distributed data with no need for individual nodes to share data. Unfortunately, it has also been shown that adversaries may be able to extract local data contents off model parameters transmitted during federated learning. A recent solution based on the secure aggregation primitive enabled privacy-preserving federated learning, but at the expense of significant extra communication/computational resources. In this paper, we propose a low-complexity scheme that provides data privacy using substantially reduced communication/computational resources relative to the existing secure solution. The key idea behind the suggested scheme is to design the topology of secret-sharing nodes as a sparse random graph instead of the complete graph corresponding to the existing solution. We first obtain the necessary and sufficient condition on the graph to guarantee both reliability and privacy. We then suggest using the Erdős-Rényi graph in particular and provide theoretical guarantees on the reliability/privacy of the proposed scheme. Through extensive real-world experiments, we demonstrate that our scheme, using only $20 \sim 30\%$ of the resources required in the conventional scheme, maintains virtually the same levels of reliability and data privacy in practical federated learning systems.

Motivation & Objective

• Motivate privacy in federated learning where local data should remain private during model aggregation.
• Develop a secure aggregation protocol with reduced communication and computation by using sparse graph topologies.
• Establish necessary/sufficient graph-based conditions for reliability and privacy in CCESA.
• Provide theoretical performance guarantees and empirical validation on real datasets.

Proposed method

• Replace the complete graph in secure aggregation with a low-degree graph to reduce resource use.
• Use an assignment graph G where edges indicate which clients share keys and secret shares.
• Derive necessary/sufficient conditions on G to ensure reliability and privacy (Theorems 1 and 2).
• Adopt an Erdős–Rényi graph G(n,p) and derive p thresholds to achieve high-probability reliability and privacy (Theorems 3–4).
• Provide finite-n bounds for reliability/privacy (Theorems 5–6) and compare complexity with existing SA and Turbo-aggregate.
• Experimentally validate CCESA against SA on AT&T face data and CIFAR-10, measuring running time, reliability, and privacy under model inversion and membership inference attacks.

Experimental results

Research questions

• RQ1 How can secure aggregation be made more scalable in federated learning without sacrificing privacy?
• RQ2 What graph topology guarantees reliable and private aggregation with sparse connections?
• RQ3 What are the theoretical thresholds (in p, n, q) needed for CCESA to be reliable and private?
• RQ4 How does CCESA perform empirically in terms of time, reliability, and privacy versus existing SA?
• RQ5 What are the practical resource savings when using CCESA on real datasets?

Key findings

• CCESA achieves similar reliability and privacy to SA while using significantly fewer resources (communication/computation).
• Using an Erdős–Rényi graph topology, CCESA can reduce client communication and key agreement workload by a factor of at least O(√(n/log n)) relative to SA.
• Theoretical thresholds show CCESA(n,p) is a.a.s. reliable and private when p exceeds a problem-dependent p*, which decreases with n.
• Finite-n bounds show extremely small privacy error (below 10^-40) and reliable error below 10^-2 per round under reasonable dropout rates.
• Empirical results on AT&T and CIFAR-10 demonstrate CCESA with 40–60% resource savings while maintaining comparable accuracy and privacy under attacks.
• CCESA running time is significantly lower than SA across tested scales, due to fewer shares and key agreements.