[论文解读] Differentially Private Distributed Computation via Public-Private Communication Networks

Lei Wang, Yang Liu+2 more|arXiv (Cornell University)|Jan 5, 2021

Privacy-Preserving Technologies in Data参考文献 39被引用 4

一句话总结

本文提出了一种用于差分隐私分布式计算的公开-私密通信网络框架，通过在稀疏私密网络上使用多播隐私保护/求和一致性（PPSC）机制注入噪声，同时保持求和一致性。该方法可实现任意所需的差分隐私级别，并以任意概率保证平均一致性、线性方程和凸优化的准确性，通过自适应噪声和计算步长实现。

ABSTRACT

This paper studies the problem of multi-agent computation under the differential privacy requirement of the agents' local datasets against eavesdroppers having node-to-node communications. We first propose for the network equipped with public-private networks. The private network is sparse and not even necessarily connected, over which communications are encrypted and secure along with the intermediate node states; the public network is connected and may be dense, over which communications are allowed to be public. In this setting, we propose a multi-gossip PPSC mechanism over the private network, where at each step, randomly selected node pairs update their states in such a way that they are shuffled with random noise while maintaining summation consistency. We show that this mechanism can achieve any desired differential privacy level with any prescribed probability. Next, we embed this mechanism in distributed computing processes, and propose privacy-guarantee protocols for three basic computation tasks, where an adaptive mechanism adjusts the amount of noise injected in PPSC steps for privacy protection, and the number of regular computation steps for accuracy guarantee. For average consensus, we develop a PPSC-Gossip averaging consensus algorithm by utilizing the multi-gossip PPSC mechanism for privacy encryption before an averaging consensus algorithm over the public network for local computations. For network linear equations and distributed convex optimization, we develop two respective distributed computing protocols by following the PPSC-Gossip averaging consensus algorithm with an additional projection or gradient descent step within each step of computation. Given any privacy and accuracy requirements, it is shown that all three proposed protocols can compute their corresponding problems with the desired computation accuracy, while achieving the desired differential privacy.

研究动机与目标

• 解决在存在窃听威胁的情况下，多智能体分布式计算中保护敏感本地数据集的挑战。
• 消除分布式系统中计算准确性、收敛效率与差分隐私之间的权衡。
• 设计一种协议，确保在基本分布式计算任务中具备强隐私保障和高准确性。
• 通过自适应噪声注入和计算调度，实现任意隐私预算和准确性水平。

提出的方法

• 引入公开-私密通信网络：公开网络为密集且连通的通信网络，用于公开通信；私密网络为稀疏且可能不连通的网络，用于安全加密通信。
• 提出多播PPSC机制，在保持网络中节点状态总和不变的同时向节点状态注入随机噪声。
• 在PPSC步骤中采用自适应噪声缩放，以实现任意所需差分隐私级别并满足指定概率。
• 将PPSC机制嵌入平均一致性、网络线性方程和分布式凸优化的分布式算法中。
• 在PPSC更新后集成投影或梯度下降步骤，以在优化和方程求解任务中保持准确性。
• 采用概率分析确保所有节点足够频繁地更新，从而在窃听攻击下实现强差分隐私。

实验结果

研究问题

• RQ1 分布式系统是否能在窃听攻击下同时实现高计算准确性和强差分隐私？
• RQ2 如何设计噪声注入机制，以在多智能体系统中保持求和一致性并确保差分隐私？
• RQ3 能否通过自适应噪声调度消除分布式计算中隐私与准确性之间的权衡？
• RQ4 何种网络架构可实现最小化对收敛性和准确性影响的可证明隐私？
• RQ5 如何将现有分布式算法（一致性、线性系统、优化）扩展以通过公开-私密通信模型支持差分隐私？

主要发现

• PPSC机制通过注入足够噪声并确保足够多的递归更新，可实现任意所需差分隐私级别，并以任意概率满足要求。
• 该机制保持了网络中总和的一致性，从而在注入噪声的情况下仍能实现准确计算。
• PPSC-Gossip平均一致性算法在私密网络上完成噪声保护的状态更新后，可在公开网络上实现准确的平均值计算。
• 针对网络线性方程和分布式凸优化所提出的协议，通过自适应噪声控制和迭代优化，同时实现了期望的准确性和差分隐私。
• 数值结果表明，在相同隐私预算下，分类任务的学习准确率优于现有方法。
• 理论分析证实，所提协议满足差分隐私要求，且通过足够多的计算步骤，误差界可被任意缩小。