[论文解读] Quantifying Privacy Risks of Prompts in Visual Prompt Learning

Yixin Wu, Rui Wen+5 more|arXiv (Cornell University)|Oct 18, 2023

Adversarial Robustness in Machine LearningComputer Science被引用 3

一句话总结

本文对提示学习中的视觉提示进行了首次全面的隐私风险评估，表明提示容易受到属性推断攻击和成员推断攻击的影响。尽管参数量减少，提示仍会泄露其训练数据的敏感信息，即使在较宽松的假设条件下，攻击仍可能成功，凸显了提示即服务部署中的关键隐私风险。

ABSTRACT

Large-scale pre-trained models are increasingly adapted to downstream tasks through a new paradigm called prompt learning. In contrast to fine-tuning, prompt learning does not update the pre-trained model's parameters. Instead, it only learns an input perturbation, namely prompt, to be added to the downstream task data for predictions. Given the fast development of prompt learning, a well-generalized prompt inevitably becomes a valuable asset as significant effort and proprietary data are used to create it. This naturally raises the question of whether a prompt may leak the proprietary information of its training data. In this paper, we perform the first comprehensive privacy assessment of prompts learned by visual prompt learning through the lens of property inference and membership inference attacks. Our empirical evaluation shows that the prompts are vulnerable to both attacks. We also demonstrate that the adversary can mount a successful property inference attack with limited cost. Moreover, we show that membership inference attacks against prompts can be successful with relaxed adversarial assumptions. We further make some initial investigations on the defenses and observe that our method can mitigate the membership inference attacks with a decent utility-defense trade-off but fails to defend against property inference attacks. We hope our results can shed light on the privacy risks of the popular prompt learning paradigm. To facilitate the research in this direction, we will share our code and models with the community.

研究动机与目标

• 调查提示学习中的视觉提示是否会从其训练数据中泄露专有信息。
• 评估针对提示作为新型隐私攻击面的属性推断和成员推断攻击的可行性。
• 评估现有攻击方法在提示的输入级隐私风险上的适用性，该风险与模型级漏洞不同。
• 探索初步的防御策略，特别是基于噪声的缓解方法，以保护提示免受推断攻击。

提出的方法

• 采用经调整的既定属性推断攻击方法——使用在影子提示上训练的元分类器——来推断提示训练数据集的敏感属性。
• 应用成员推断攻击（基于神经网络、基于度量和基于梯度）以确定给定图像是否属于提示的训练数据集。
• 使用与目标提示相同的训练数据分布训练多个影子提示，以构建攻击模型。
• 通过向提示添加高斯噪声并测量效用和攻击成功率，评估防御机制。
• 使用标准视觉基准（如 CUB-200-2011、Cifar-10）在真实场景下评估攻击和防御性能。
• 共享代码和模型，以支持提示隐私研究的可复现性和进一步探索。

Figure 1 : Overview of prompt usage and inference attacks. The prompt is a pixel patch. The prompted image is an original image with an added prompt. Property inference infers sensitive properties of the target prompt’s training dataset that the PaaS provider does not intend to disclose. Membership

实验结果

研究问题

• RQ1 攻击者在多大程度上能够推断出用于创建视觉提示的训练数据的敏感属性？
• RQ2 成员推断攻击能否确定特定图像是否属于提示的训练数据集？
• RQ3 在参数量少且具有输入级特性的前提下，现有隐私攻击在提示上的有效性如何？
• RQ4 基于噪声的防御是否能在保持提示效用的同时缓解成员推断攻击？
• RQ5 为何当前防御方法无法有效保护提示学习范式下的属性推断攻击？

主要发现

• 视觉提示易受属性推断攻击影响，攻击者仅需少量影子提示即可成功推断出训练数据集的敏感属性。
• 即使在攻击者对训练数据分布访问受限等较宽松的假设条件下，对提示的成员推断攻击仍具有效性。
• 向提示添加高斯噪声可在防御效果与模型效用之间实现合理权衡，从而缓解成员推断攻击。
• DP-SGD 虽能提供一定保护，但无法有效保留提示效用，且对属性推断攻击的防护不足。
• 所提出的防御策略无法推广至属性推断攻击，表明当前针对输入级隐私风险的防御方法存在关键缺口。
• 结果表明，尽管提示学习具有高效性，但其在输入级别引入了新的隐私风险，而现有防御手段尚不足以有效应对。

Figure 2 : Overview of visual prompt learning (VPL). We learn an input prompt via back-propagation [ 4 ] at the input transformation stage. We apply hard-coded mapping [ 13 ] to map the pre-trained model’s outputs into the target labels at the output transformation stage.