[论文解读] Local Differential Privacy based Federated Learning for Internet of Things

Yang Zhao, Jun Zhao+6 more|arXiv (Cornell University)|Apr 19, 2020

Privacy-Preserving Technologies in Data参考文献 57被引用 38

一句话总结

本文将本地差分隐私与联邦学习相结合用于IoV众包，提出新的LDP机制来扰动梯度，并提出LDP-FedSGD算法在保护隐私的同时训练模型并降低通信成本。

ABSTRACT

Internet of Vehicles (IoV) is a promising branch of the Internet of Things. IoV simulates a large variety of crowdsourcing applications such as Waze, Uber, and Amazon Mechanical Turk, etc. Users of these applications report the real-time traffic information to the cloud server which trains a machine learning model based on traffic information reported by users for intelligent traffic management. However, crowdsourcing application owners can easily infer users' location information, which raises severe location privacy concerns of the users. In addition, as the number of vehicles increases, the frequent communication between vehicles and the cloud server incurs unexpected amount of communication cost. To avoid the privacy threat and reduce the communication cost, in this paper, we propose to integrate federated learning and local differential privacy (LDP) to facilitate the crowdsourcing applications to achieve the machine learning model. Specifically, we propose four LDP mechanisms to perturb gradients generated by vehicles. The Three-Outputs mechanism is proposed which introduces three different output possibilities to deliver a high accuracy when the privacy budget is small. The output possibilities of Three-Outputs can be encoded with two bits to reduce the communication cost. Besides, to maximize the performance when the privacy budget is large, an optimal piecewise mechanism (PM-OPT) is proposed. We further propose a suboptimal mechanism (PM-SUB) with a simple formula and comparable utility to PM-OPT. Then, we build a novel hybrid mechanism by combining Three-Outputs and PM-SUB.

研究动机与目标

• 在IoV众包应用中激发对隐私的关注以及对隐私保护协同学习的需求。
• 开发一个在梯度层面通过本地差分隐私保护用户数据的FL框架。
• 提出新型LDP机制和一个LDP-FedSGD算法，在隐私与模型效用及通信效率之间取得平衡。

提出的方法

• 提出四种数值数据的本地DP机制（Three-Outputs、PM-OPT、PM-SUB、HM-TP），以在受控隐私预算的前提下扰动梯度。
• 引入LDP-FedSGD算法，其中车辆计算本地梯度，应用LDP机制，云端服务器聚合带噪梯度以更新全局模型。
• 将连续输出离散化以便编码并在不牺牲效用的前提下降低通信成本。
• 比较扰动放置和隐私粒度，以在带有诚实但好奇的聚合方的分布式扰动设置中证明ULDP的合理性。
• 对相对于现有LDP机制的效用提升进行理论与实证评估。

实验结果

研究问题

• RQ1 如何在IoV中将本地差分隐私有效地融入联邦学习，以保护敏感的用户信息？
• RQ2 在FL中对梯度扰动而言，哪些LDP机制在隐私-效用权衡方面表现最佳？
• RQ3 一种混合LDP机制能否在物联网联邦学习中改善在不同隐私预算下的性能？
• RQ4 对LDP机制的连续输出进行离散化是否在降低通信成本的同时维持效用？

主要发现

• Three-Outputs在ε>ln 2（约0.69）时优于Duchi；在较大ε范围内，PM-SUB优于PM和Duchi；混合式HM-TP进一步降低了最坏情况方差。
• PM-SUB在表达式更简单的前提下提供与PM-OPT相当或更好的效用。
• 离线离散化后处理在降低通信成本的同时保留效用。
• 在真实数据和合成数据上的实验结果表明，在LDP-FedSGD下，均值频率估计与经验风险最小化任务的准确性得到提高。
• 在广泛的ε值范围内，所提出的机制相比现有LDP机制具有更高的效用。