[论文解读] Label Leakage and Protection from Forward Embedding in Vertical Federated Learning

Jiankai Sun, Xin Yang+2 more|arXiv (Cornell University)|Mar 2, 2022

Privacy-Preserving Technologies in Data被引用 11

一句话总结

本文识别并缓解了垂直联邦学习（vFL）中前向中间嵌入导致的标签泄露问题，提出一种谱攻击方法，即使在已有基于梯度的保护机制下，仍能推断出私有标签。本文提出一种防御方法，在标签方最小化嵌入与标签之间的距离相关性，将攻击AUC降低至约0.5，同时保持模型性能。

ABSTRACT

Vertical federated learning (vFL) has gained much attention and been deployed to solve machine learning problems with data privacy concerns in recent years. However, some recent work demonstrated that vFL is vulnerable to privacy leakage even though only the forward intermediate embedding (rather than raw features) and backpropagated gradients (rather than raw labels) are communicated between the involved participants. As the raw labels often contain highly sensitive information, some recent work has been proposed to prevent the label leakage from the backpropagated gradients effectively in vFL. However, these work only identified and defended the threat of label leakage from the backpropagated gradients. None of these work has paid attention to the problem of label leakage from the intermediate embedding. In this paper, we propose a practical label inference method which can steal private labels effectively from the shared intermediate embedding even though some existing protection methods such as label differential privacy and gradients perturbation are applied. The effectiveness of the label attack is inseparable from the correlation between the intermediate embedding and corresponding private labels. To mitigate the issue of label leakage from the forward embedding, we add an additional optimization goal at the label party to limit the label stealing ability of the adversary by minimizing the distance correlation between the intermediate embedding and corresponding private labels. We conducted massive experiments to demonstrate the effectiveness of our proposed protection methods.

研究动机与目标

• 识别尽管已有梯度保护机制，vFL中的中间嵌入仍可能泄露私有标签。
• 证明即使在应用标签差分隐私和梯度扰动的情况下，仍可利用谱攻击从前向嵌入中推断出标签。
• 提出一种新颖的防御方法，使标签方最小化剪切层嵌入与私有标签之间的距离相关性。
• 评估该防御方法与现有基于梯度的隐私方法的兼容性与有效性。

提出的方法

• 提出一种谱攻击方法，用于在两方vFL中从前向中间嵌入推断私有标签。
• 在标签方引入额外的优化目标，以最小化剪切层嵌入与对应私有标签之间的距离相关性。
• 使用距离相关性作为可微分度量，量化并减少嵌入与标签之间的统计依赖性。
• 采用超参数 αd 控制正则化项的强度，平衡隐私保护与模型性能。
• 通过与Marvell（梯度扰动）和Label DP等现有防御机制结合，验证了所提方法的兼容性。
• 在真实世界数据集上进行大量实验，评估攻击成功率与防御有效性。

实验结果

研究问题

• RQ1 即使在基于梯度的保护机制存在的情况下，是否仍能有效从前向中间嵌入中推断出私有标签？
• RQ2 中间嵌入与私有标签之间的相关性在导致标签泄露中起什么作用？
• RQ3 标签方如何在不降低模型性能的前提下，最小化从前向嵌入泄露标签的风险？
• RQ4 所提出的防御方法能否有效与现有基于梯度的隐私机制结合？
• RQ5 在不同正则化强度下，模型性能与隐私保护之间的权衡关系如何？

主要发现

• 谱攻击在未受保护的嵌入上实现了接近1.0的标签推断AUC，表明其在窃取私有标签方面具有极高有效性。
• 当 αd = 0.002 时，所提防御将攻击AUC降低至约0.5（随机猜测水平），表明具备强大的隐私保护能力。
• 当 αd = 0.002 时，模型性能（以测试AUC衡量）几乎未受影响（与原始模型相差不超过0.01），表明性能下降可忽略。
• 该防御与梯度扰动方法（如Marvell）兼容，可同时防范前向嵌入和基于梯度的标签泄露。
• 敏感性分析表明，αd ∈ [0.002, 0.005] 范围内可实现隐私保护（AUC ≈ 0.5）与模型性能（测试AUC基本不变）的良好平衡。
• 隐私成本体现为标签方需重新学习以降低相关性，导致计算负载增加，但该开销在可接受范围内。