[Paper Review] Differentially Private Distributed Computation via Public-Private Communication Networks

Lei Wang, Yang Liu+2 more|arXiv (Cornell University)|Jan 5, 2021

Privacy-Preserving Technologies in Data39 references4 citations

TL;DR

This paper proposes a public-private communication network framework for differentially private distributed computation, using a multi-gossip Privacy-Preserving/Summation-Consistent (PPSC) mechanism over a sparse private network to inject noise while preserving sum consistency. It achieves any desired differential privacy level with arbitrary probability and guarantees accuracy for average consensus, linear equations, and convex optimization via adaptive noise and computation steps.

ABSTRACT

This paper studies the problem of multi-agent computation under the differential privacy requirement of the agents' local datasets against eavesdroppers having node-to-node communications. We first propose for the network equipped with public-private networks. The private network is sparse and not even necessarily connected, over which communications are encrypted and secure along with the intermediate node states; the public network is connected and may be dense, over which communications are allowed to be public. In this setting, we propose a multi-gossip PPSC mechanism over the private network, where at each step, randomly selected node pairs update their states in such a way that they are shuffled with random noise while maintaining summation consistency. We show that this mechanism can achieve any desired differential privacy level with any prescribed probability. Next, we embed this mechanism in distributed computing processes, and propose privacy-guarantee protocols for three basic computation tasks, where an adaptive mechanism adjusts the amount of noise injected in PPSC steps for privacy protection, and the number of regular computation steps for accuracy guarantee. For average consensus, we develop a PPSC-Gossip averaging consensus algorithm by utilizing the multi-gossip PPSC mechanism for privacy encryption before an averaging consensus algorithm over the public network for local computations. For network linear equations and distributed convex optimization, we develop two respective distributed computing protocols by following the PPSC-Gossip averaging consensus algorithm with an additional projection or gradient descent step within each step of computation. Given any privacy and accuracy requirements, it is shown that all three proposed protocols can compute their corresponding problems with the desired computation accuracy, while achieving the desired differential privacy.

Motivation & Objective

• To address the challenge of protecting sensitive local datasets in multi-agent distributed computation under eavesdropping threats.
• To eliminate the trade-off between computational accuracy, convergence efficiency, and differential privacy in distributed systems.
• To design a protocol that ensures both strong privacy guarantees and high accuracy for fundamental distributed computation tasks.
• To enable arbitrary privacy budgets and accuracy levels through adaptive noise injection and computation scheduling.

Proposed method

• Introduces a public-private communication network: public network is dense and connected for public communication, private network is sparse and potentially disconnected for secure, encrypted communication.
• Develops a Multi-Gossiping PPSC mechanism that injects random noise into node states while preserving the sum of states across the network.
• Uses adaptive noise scaling in PPSC steps to achieve any desired differential privacy level with prescribed probability.
• Embeds the PPSC mechanism into distributed algorithms for average consensus, network linear equations, and distributed convex optimization.
• Integrates projection or gradient descent steps after PPSC updates to maintain accuracy in optimization and equation-solving tasks.
• Employs probabilistic analysis to ensure that all nodes update sufficiently often, enabling strong differential privacy under eavesdropping.

Experimental results

Research questions

• RQ1 Can a distributed system achieve both high computational accuracy and strong differential privacy under eavesdropping attacks?
• RQ2 How can noise injection be designed to preserve sum consistency while ensuring differential privacy in multi-agent systems?
• RQ3 Can the trade-off between privacy and accuracy in distributed computation be eliminated through adaptive noise scheduling?
• RQ4 What network architecture enables provable privacy with minimal impact on convergence and accuracy?
• RQ5 How can existing distributed algorithms (consensus, linear systems, optimization) be extended to support differential privacy via a public-private communication model?

Key findings

• The PPSC mechanism achieves any desired differential privacy level with arbitrary probability by injecting sufficient noise and ensuring enough recursive updates.
• The mechanism maintains summation consistency across the network, enabling accurate computation despite noise injection.
• The PPSC-Gossip averaging consensus algorithm achieves accurate average computation over the public network after noise-protected state updates on the private network.
• The proposed protocols for network linear equations and distributed convex optimization achieve both desired accuracy and differential privacy through adaptive noise control and iterative refinement.
• Numerical results show improved learning accuracy in classification tasks compared to existing methods under the same privacy budget.
• Theoretical analysis confirms that the protocols satisfy differential privacy with user-defined parameters, and the error bound can be made arbitrarily small with sufficient computation steps.