[论文解读] Communication-Computation Efficient Secure Aggregation for Federated Learning

Beongjun Choi, Jy-yong Sohn+2 more|arXiv (Cornell University)|Dec 10, 2020

Privacy-Preserving Technologies in Data参考文献 39被引用 48

一句话总结

CCESA 用稀疏分配图（Erdős–Rényi）替代安全聚合的完全图秘密共享，在通信和计算成本大幅降低的同时实现类似的可靠性和隐私。作者给出理论条件和实证结果，显示在真实攻击下资源节省达 60-70%，并且隐私性强。

ABSTRACT

Federated learning has been spotlighted as a way to train neural networks using distributed data with no need for individual nodes to share data. Unfortunately, it has also been shown that adversaries may be able to extract local data contents off model parameters transmitted during federated learning. A recent solution based on the secure aggregation primitive enabled privacy-preserving federated learning, but at the expense of significant extra communication/computational resources. In this paper, we propose a low-complexity scheme that provides data privacy using substantially reduced communication/computational resources relative to the existing secure solution. The key idea behind the suggested scheme is to design the topology of secret-sharing nodes as a sparse random graph instead of the complete graph corresponding to the existing solution. We first obtain the necessary and sufficient condition on the graph to guarantee both reliability and privacy. We then suggest using the Erdős-Rényi graph in particular and provide theoretical guarantees on the reliability/privacy of the proposed scheme. Through extensive real-world experiments, we demonstrate that our scheme, using only $20 \sim 30\%$ of the resources required in the conventional scheme, maintains virtually the same levels of reliability and data privacy in practical federated learning systems.

研究动机与目标

• Motivate privacy in federated learning where local data should remain private during model aggregation.
• Develop a secure aggregation protocol with reduced communication and computation by using sparse graph topologies.
• Establish necessary/sufficient graph-based conditions for reliability and privacy in CCESA.
• Provide theoretical performance guarantees and empirical validation on real datasets.

提出的方法

• Replace the complete graph in secure aggregation with a low-degree graph to reduce resource use.
• Use an assignment graph G where edges indicate which clients share keys and secret shares.
• Derive necessary/sufficient conditions on G to ensure reliability and privacy (Theorems 1 and 2).
• Adopt an Erdős–Rényi graph G(n,p) and derive p thresholds to achieve high-probability reliability and privacy (Theorems 3–4).
• Provide finite-n bounds for reliability/privacy (Theorems 5–6) and compare complexity with existing SA and Turbo-aggregate.
• Experimentally validate CCESA against SA on AT&T face data and CIFAR-10, measuring running time, reliability, and privacy under model inversion and membership inference attacks.

实验结果

研究问题

• RQ1 How can secure aggregation be made more scalable in federated learning without sacrificing privacy?
• RQ2 What graph topology guarantees reliable and private aggregation with sparse connections?
• RQ3 What are the theoretical thresholds (in p, n, q) needed for CCESA to be reliable and private?
• RQ4 How does CCESA perform empirically in terms of time, reliability, and privacy versus existing SA?
• RQ5 What are the practical resource savings when using CCESA on real datasets?

主要发现

Graph topology	Communication cost (Client)	Communication cost (Server)	Computation cost (Client)	Computation cost (Server)
CCESA	Erdős-Rényi graph	O(√(n log n)+ m)	O(n√(n log n)+ mn)	O(n log n + m√(n log n))	O(mn log n) + n^2 log n
Bell et al. 2020	Harary graph	O(log n+m)	O(n log n+ mn)	O(log^2 n + m log n)	O(mn log n + n log^2 n)
SA	Complete graph	O(n+m)	O(n^2+mn)	O(n^2+mn)	O(mn^2)

• CCESA achieves similar reliability and privacy to SA while using significantly fewer resources (communication/computation).
• Using an Erdős–Rényi graph topology, CCESA can reduce client communication and key agreement workload by a factor of at least O(√(n/log n)) relative to SA.
• Theoretical thresholds show CCESA(n,p) is a.a.s. reliable and private when p exceeds a problem-dependent p*, which decreases with n.
• Finite-n bounds show extremely small privacy error (below 10^-40) and reliable error below 10^-2 per round under reasonable dropout rates.
• Empirical results on AT&T and CIFAR-10 demonstrate CCESA with 40–60% resource savings while maintaining comparable accuracy and privacy under attacks.
• CCESA running time is significantly lower than SA across tested scales, due to fewer shares and key agreements.