[论文解读] Privacy Leakage of Real-World Vertical Federated Learning.

Haiqin Weng, Juntao Zhang+4 more|arXiv (Cornell University)|Nov 18, 2020

Privacy-Preserving Technologies in Data参考文献 35被引用 27

一句话总结

本文揭示了即使在诚实但好奇的对手（遵循协议规则）下，纵向联邦学习（VFL）系统仍存在隐私泄露风险。作者设计了两种高效且非侵入式的攻击——反向求和与反向乘法攻击，能够在不影响模型准确率或偏离协议规则的前提下，从模型更新中重建私有训练数据。

ABSTRACT

Federated learning enables mutually distrusting participants to collaboratively learn a distributed machine learning model without revealing anything but the model's output. Generic federated learning has been studied extensively, and several learning protocols, as well as open-source frameworks, have been developed. Yet, their over pursuit of computing efficiency and fast implementation might diminish the security and privacy guarantees of participant's training data, about which little is known thus far. In this paper, we consider an honest-but-curious adversary who participants in training a distributed ML model, does not deviate from the defined learning protocol, but attempts to infer private training data from the legitimately received information. In this setting, we design and implement two practical attacks, reverse sum attack and reverse multiplication attack, neither of which will affect the accuracy of the learned model. By empirically studying the privacy leakage of two learning protocols, we show that our attacks are (1) effective - the adversary successfully steal the private training data, even when the intermediate outputs are encrypted to protect data privacy; (2) evasive - the adversary's malicious behavior does not deviate from the protocol specification and deteriorate any accuracy of the target model; and (3) easy - the adversary needs little prior knowledge about the data distribution of the target participant. We also experimentally show that the leaked information is as effective as the raw training data through training an alternative classifier on the leaked information. We further discuss potential countermeasures and their challenges, which we hope may lead to several promising research directions.

研究动机与目标

• 探究在诚实但好奇的对手遵循协议规则的情况下，纵向联邦学习中的隐私保障是否被破坏。
• 设计可实际应用的攻击方法，利用中间模型更新重建私有训练数据，同时不改变模型准确率。
• 评估这些攻击在中间输出被加密时的有效性。
• 评估泄露的信息对下游学习任务的有用性是否与原始训练数据相当。
• 识别防御此类协议合规隐私攻击的挑战，并为未来隐私保护型VFL协议提供研究方向。

提出的方法

• 设计反向求和攻击，通过反转纵向联邦学习中模型梯度或更新的聚合过程来恢复输入特征。
• 开发反向乘法攻击，利用某些VFL协议中模型更新的乘法结构来推断私有数据。
• 在真实世界的VFL框架中实现这两种攻击，以评估其在标准协议约束下的可行性与有效性。
• 在实验中使用加密的中间输出，测试仅靠加密是否足以防止数据重建。
• 在泄露数据上训练代理分类器，以验证其与原始训练数据相比的实用性。
• 在对数据分布仅有最少先验知识的条件下分析攻击成功率，证明其对辅助信息的依赖程度极低。

实验结果

研究问题

• RQ1 诚实但好奇的对手能否仅从纵向联邦学习中合法的模型更新中重建私有训练数据？
• RQ2 当中间模型输出被加密时，这些攻击是否仍具有效性？
• RQ3 对手成功发动这些攻击所需的数据分布先验知识有多少？
• RQ4 这些攻击泄露的信息是否足以训练出与使用原始数据训练的分类器性能相当的模型？
• RQ5 防御此类协议合规隐私攻击的根本挑战是什么？

主要发现

• 反向求和与反向乘法攻击即使在中间输出被加密的情况下，也能成功从模型更新中重建纵向联邦学习中的私有训练数据。
• 这些攻击未偏离协议规范，且不会降低最终模型的准确率。
• 攻击者仅需极少的数据分布先验知识，使攻击具有实际可行性并广泛适用。
• 泄露数据在训练下游分类器方面与原始训练数据具有同等有效性，在基准数据集上表现相当。
• 现有中间输出的加密手段不足以防止此类攻击下的隐私泄露。
• 研究结果揭示了当前VFL安全模型中的关键漏洞，并呼吁开发超越简单加密的新一代隐私保护机制。