[Paper Review] Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data

Shenglai Zeng, Jiankun Zhang+8 more|arXiv (Cornell University)|Jun 20, 2024

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes SAGE, a two-stage synthetic data generation framework to mitigate privacy risks in Retrieval-Augmented Generation (RAG) by replacing sensitive retrieval data with high-utility, privacy-preserving synthetic data. The method first extracts and generates key attributes from original data, then refines it via an agent-based iterative process, achieving performance comparable to real data while drastically reducing privacy leakage in both targeted and untargeted attacks.

ABSTRACT

Retrieval-augmented generation (RAG) enhances the outputs of language models by integrating relevant information retrieved from external knowledge sources. However, when the retrieval process involves private data, RAG systems may face severe privacy risks, potentially leading to the leakage of sensitive information. To address this issue, we propose using synthetic data as a privacy-preserving alternative for the retrieval data. We propose SAGE, a novel two-stage synthetic data generation paradigm. In the stage-1, we employ an attribute-based extraction and generation approach to preserve key contextual information from the original data. In the stage-2, we further enhance the privacy properties of the synthetic data through an agent-based iterative refinement process. Extensive experiments demonstrate that using our synthetic data as the retrieval context achieves comparable performance to using the original data while substantially reducing privacy risks. Our work takes the first step towards investigating the possibility of generating high-utility and privacy-preserving synthetic data for RAG, opening up new opportunities for the safe application of RAG systems in various domains.

Motivation & Objective

• To address privacy risks in RAG systems when retrieval data contains sensitive information.
• To develop a synthetic data generation pipeline that preserves essential information for downstream task utility.
• To enhance privacy protection without compromising retrieval quality or introducing significant inference overhead.
• To explore the feasibility of using synthetic data as a safe, surrogate replacement for real retrieval data in RAG.
• To evaluate the robustness of synthetic data against both targeted and untargeted privacy attacks.

Proposed method

• A two-stage synthetic data generation pipeline: stage-1 uses attribute-based extraction and generation to preserve key contextual information from original data.
• Stage-2 employs an agent-based iterative refinement process to further enhance privacy by removing sensitive elements while maintaining utility.
• The attribute-based generation stage uses LLMs to identify and reconstruct core information, such as medical conditions, symptoms, and treatments, from original patient records.
• The agent-based refinement uses multiple LLM agents to iteratively evaluate and modify the synthetic data, ensuring privacy hardening through self-consistency and adversarial testing.
• The framework is evaluated using real-world datasets like HealthCareMagic, with performance measured via standard RAG metrics and privacy leakage via targeted and untargeted attack benchmarks.
• The method is designed to be agnostic to the downstream LLM, allowing integration with various LLMs without architectural changes.

Figure 1: An illustration for RAG with synthetic data.

Experimental results

Research questions

• RQ1 Can synthetic data be generated to maintain high utility for RAG while effectively mitigating privacy leakage from sensitive retrieval data?
• RQ2 How does the two-stage synthetic data generation pipeline—attribute-based generation followed by agent-based refinement—affect performance and privacy?
• RQ3 What is the impact of the number of attributes used in the generation process on utility and privacy?
• RQ4 How robust is the synthetic data pipeline against increasing numbers of retrieved documents in RAG?
• RQ5 How does the choice of LLM (e.g., GPT-3.5 vs. GPT-4) affect the privacy and quality of the generated synthetic data?

Key findings

• Using SAGE-generated synthetic data in RAG achieves performance comparable to using original data, with minimal drop in retrieval and generation quality across multiple benchmarks.
• The agent-based refinement stage significantly reduces privacy leakage, achieving near-zero exposure in targeted attacks even when retrieving multiple documents.
• When using Llama3-Chat-8b for stage-1 generation, no privacy leakage was observed due to poor data quality, while GPT-4 showed lower leakage than GPT-3.5, likely due to stronger safety filtering.
• Increasing the number of attributes improves performance up to a point, but diminishing returns are observed beyond an optimal number, suggesting a need for careful hyperparameter tuning.
• The pipeline remains robust against increasing retrieval document counts (k), with consistently low privacy leakage across k=1 to k=10, unlike stage-1 data alone which shows amplified risks.
• The method effectively mitigates both targeted and untargeted attacks, demonstrating that synthetic data can serve as a viable privacy-preserving alternative to real retrieval data in RAG.

Figure 2: Pipeline of generating synthetic data.