[Paper Review] Large Language Models are Advanced Anonymizers

Robin Staab, Mark Vero+2 more|arXiv (Cornell University)|Feb 21, 2024

Topic Modeling4 citations

TL;DR

This paper proposes a novel LLM-based adversarial anonymization framework that uses feedback from a powerful adversary LLM to iteratively refine text anonymization, significantly improving privacy and utility over existing tools. The method outperforms commercial anonymizers like Azure and GPT-4-based baselines in both human preference studies and automated evaluations, even enabling smaller quantized LLMs to match the performance of larger models.

ABSTRACT

Recent privacy research on large language models (LLMs) has shown that they achieve near-human-level performance at inferring personal data from online texts. With ever-increasing model capabilities, existing text anonymization methods are currently lacking behind regulatory requirements and adversarial threats. In this work, we take two steps to bridge this gap: First, we present a new setting for evaluating anonymization in the face of adversarial LLM inferences, allowing for a natural measurement of anonymization performance while remedying some of the shortcomings of previous metrics. Then, within this setting, we develop a novel LLM-based adversarial anonymization framework leveraging the strong inferential capabilities of LLMs to inform our anonymization procedure. We conduct a comprehensive experimental evaluation of adversarial anonymization across 13 LLMs on real-world and synthetic online texts, comparing it against multiple baselines and industry-grade anonymizers. Our evaluation shows that adversarial anonymization outperforms current commercial anonymizers both in terms of the resulting utility and privacy. We support our findings with a human study (n=50) highlighting a strong and consistent human preference for LLM-anonymized texts.

Motivation & Objective

• To address the growing threat of LLMs inferring personal attributes from seemingly innocuous online texts.
• To overcome the limitations of current industry-standard anonymizers, which fail to detect context-dependent, indirect privacy leaks.
• To develop a feedback-guided adversarial anonymization framework that leverages LLMs' inferential power to improve anonymization quality.
• To evaluate anonymization performance under realistic adversarial inference conditions, aligning with regulatory privacy standards.
• To explore the feasibility of distilling high-performance anonymization into smaller, efficient, and potentially edge-deployable models.

Proposed method

• The framework employs a dual LLM setup: one adversarial LLM performs iterative attribute inference on the current text version.
• Based on the inference, a separate anonymizer LLM modifies the text by removing, obfuscating, or generalizing privacy-revealing cues.
• The process runs in multiple rounds, with each iteration refining the text to reduce inference success while preserving utility.
• Anonymization is evaluated under an adversarial inference setting, where the final output is tested against a strong LLM adversary for attribute prediction.
• The method uses synthetic data and demonstrations from larger models (e.g., Llama3.1-70B) to fine-tune smaller models (e.g., Llama3.1-8B) via LoRA finetuning.
• A human study (n=50) evaluates anonymized texts on readability, meaning, typicality, and usability to validate real-world performance.

Experimental results

Research questions

• RQ1 Can adversarial LLMs effectively identify context-dependent privacy leaks that traditional tools miss?
• RQ2 Does a feedback-guided, iterative LLM-based anonymization process improve privacy and utility compared to static anonymization?
• RQ3 Can smaller, quantized LLMs achieve performance comparable to larger models in adversarial anonymization?
• RQ4 How do human preferences for anonymized texts compare between LLM-based and commercial anonymizers?
• RQ5 To what extent do LLM-based and human judgments of meaning and readability align for anonymized outputs?

Key findings

• The adversarial anonymization framework significantly outperforms commercial tools like Azure and GPT-4-based anonymizers in both privacy and utility, as measured by adversarial inference accuracy and human evaluation.
• Human preference studies (n=50) show a strong and consistent preference for texts anonymized by LLMs over those processed by commercial tools across readability, meaning, and usability metrics.
• The finetuned 4-bit quantized Llama3.1-8B model achieves performance very close to the full 70B model in the privacy-utility tradeoff, demonstrating feasibility for efficient deployment.
• LLM-based judgments of meaning and readability for LLM-anonymized texts align well with human judgments, while for Azure-anonymized texts, LLMs rate them as more readable and meaningful than humans do.
• Even small, locally deployed LLMs outperform commercial anonymizers in privacy and utility, suggesting a path toward scalable, on-device privacy protection.
• The framework successfully identifies and mitigates indirect privacy leaks—such as location inference from cultural references—beyond the reach of traditional entity recognition tools.