[论文解读] Protecting User Privacy in Remote Conversational Systems: A Privacy-Preserving framework based on text sanitization

Zhigang Kan, Linbo Qiao+4 more|arXiv (Cornell University)|Jun 14, 2023

Privacy-Preserving Technologies in Data被引用 5

一句话总结

本文提出PP-TS，一种隐私保护框架，通过在将输入发送至大语言模型之前，基于用户定义的隐私类型对文本进行多轮清洗，从而保护远程对话式AI系统中的用户数据。该框架在对数据效用影响最小的情况下，实现了高达95.96%的隐私移除率（PRR），数据效用率（DUR）达92.33%，同时通过合理性检查机制增强了对逻辑推理攻击的抵抗能力。

ABSTRACT

Large Language Models (LLMs) are gaining increasing attention due to their exceptional performance across numerous tasks. As a result, the general public utilize them as an influential tool for boosting their productivity while natural language processing researchers endeavor to employ them in solving existing or new research problems. Unfortunately, individuals can only access such powerful AIs through APIs, which ultimately leads to the transmission of raw data to the models' providers and increases the possibility of privacy data leakage. Current privacy-preserving methods for cloud-deployed language models aim to protect privacy information in the pre-training dataset or during the model training phase. However, they do not meet the specific challenges presented by the remote access approach of new large-scale language models. This paper introduces a novel task, "User Privacy Protection for Dialogue Models," which aims to safeguard sensitive user information from any possible disclosure while conversing with chatbots. We also present an evaluation scheme for this task, which covers evaluation metrics for privacy protection, data availability, and resistance to simulation attacks. Moreover, we propose the first framework for this task, namely privacy protection through text sanitization. Before sending the input to remote large models, it filters out the sensitive information, using several rounds of text sanitization based on privacy types that users define. Upon receiving responses from the larger model, our framework automatically restores privacy to ensure that the conversation goes smoothly, without intervention from the privacy filter. Experiments based on real-world datasets demonstrate the efficacy of our privacy-preserving approach against eavesdropping from potential attackers.

研究动机与目标

• 解决用户通过API与远程大语言模型交互时日益增长的隐私泄露风险。
• 识别在远程模型访问场景中保护非结构化自然语言输入所面临的独特挑战。
• 设计一个涵盖隐私保护、数据可用性及对仿真攻击抵抗能力的综合性评估方案。
• 提出一种新颖框架，对用户输入进行清洗，并自动在模型响应中恢复隐私信息，以维持对话连贯性。

提出的方法

• 基于用户定义的隐私类型，对远程模型调用前的输入文本进行多轮文本清洗，以过滤敏感信息。
• 利用上下文学习技术程序化检测并移除隐私信息，提升检测能力，超越字面匹配的局限。
• 集成合理性检查机制，减少清洗后文本中的语义矛盾，防止推理攻击。
• 通过反向清洗过程自动在模型响应中恢复隐私信息，以保持对话连贯性。
• 采用三段式评估方案：隐私移除率（PRR）、数据效用率（DUR），以及对字面和逻辑推理攻击的抵抗能力。
• 使用精确率、召回率和F1值指标评估清洗后事件抽取任务的性能。

实验结果

研究问题

• RQ1 所提出的框架在抵御基于字面和逻辑推理的仿真攻击方面效果如何？
• RQ2 文本清洗在多大程度上影响了用户输入和模型响应的数据效用与语义连贯性？
• RQ3 合理性检查机制在增强对推理攻击的抵抗能力方面发挥何种作用？
• RQ4 该框架如何在无需人工干预的情况下维持隐私过滤过程中的对话流？
• RQ5 在隐私保护与下游任务性能（如事件抽取）之间存在何种权衡？

主要发现

• PP-TS框架实现了95.96%的隐私移除率（PRR），表明指定隐私类型在输入文本中几乎被完全清除。
• 数据效用率（DUR）达到92.33%，表明清洗后输入数据的效用得到良好保留。
• 若移除合理性检查机制，DUR将下降至81.67%，证实其在维持语义连贯性及抵抗推理攻击方面具有关键作用。
• 该框架在事件抽取任务中保持高性能，触发词F1值为48.91，论元F1值为21.00，表明与原始性能相比退化程度极小。
• 在抵御程序化攻击方面，该框架优于基线方法，此类攻击的检测能力弱于人工驱动的攻击。
• 引入合理性检查机制后，触发词F1值提升1.71点，论元F1值提升1.98点，表明下游任务性能获得可衡量的提升。