[Paper Review] EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs

Lin Guo, Wenyue Hua+4 more|arXiv (Cornell University)|Feb 8, 2024

DNA and Biological ComputingBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

EmojiCrypt is a privacy-preserving prompt obfuscation technique that encrypts sensitive user inputs in large language model (LLM) interactions by converting them into sequences of emojis, ensuring the LLM can still generate accurate outputs while rendering the original data indecipherable to humans and the LLM provider. It achieves comparable or improved task performance without model fine-tuning, demonstrating strong privacy protection with minimal performance degradation.

ABSTRACT

Cloud-based Large Language Models (LLMs) such as ChatGPT have become increasingly integral to daily operations. Nevertheless, they also introduce privacy concerns: firstly, numerous studies underscore the risks to user privacy posed by jailbreaking cloud-based LLMs; secondly, the LLM service providers have access to all user data, which deters individuals from confidently utilizing such services. To address such concerns, we propose a simple yet effective paradigm, EmojiPrompt, to protect user privacy. At its core, EmojiPrompt performs generative transformation, obfuscating private data within prompts with linguistic and non-linguistic elements before submitting them to cloud-based LLMs. We evaluate EmojiPrompt's performance across 8 datasets from various domains. We also propose simulated inference attacks to assess EmojiPrompt's ability to preserve user privacy. The results demonstrate that EmojiPrompt effectively obfuscates user private data, while largely maintaining, or even enhancing, performances compared to the unobfuscated version. Furthermore, EmojiPrompt's atomic-level obfuscation allows it to function exclusively with cloud-based LLMs. For source code, please refer to: https://github.com/agiresearch/EmojiCrypt.

Motivation & Objective

• To address privacy risks in cloud-based LLMs where sensitive user data is exposed to service providers despite encrypted transmission and storage.
• To develop a method that hides sensitive information in prompts from both human observers and the LLM itself, without compromising output quality.
• To enable secure, private communication with LLMs by transforming personal or confidential data into semantically obscured emoji sequences that retain task-relevant meaning.
• To evaluate whether encrypted prompts can maintain high task accuracy while resisting reconstruction of original sensitive content.
• To demonstrate that prompt encryption can be effective without requiring model fine-tuning or altering the LLM's inference process.

Proposed method

• The method converts sensitive elements in user prompts—such as names, product titles, or personal attributes—into sequences of emojis using a pre-trained LLM as an encoder.
• The LLM encoder maps natural language inputs into emoji-based representations that are semantically meaningful to the LLM but unintelligible to humans.
• The obfuscated emoji prompts are then sent to the target LLM (e.g., GPT-4 or Gemini-Pro) for downstream task execution without any fine-tuning.
• A decryption evaluation is performed by prompting a separate LLM (e.g., GPT-4) to reconstruct the original text from the emoji sequences, measuring similarity via cosine similarity.
• Robustness is assessed by comparing the similarity between reconstructed text and the original, with a safety threshold computed using random samples from the same dataset to establish a baseline for irrelevance.
• The approach is evaluated across three tasks: e-commerce recommendation (Amazon Beauty), sentiment analysis (IMDB), and financial status classification (Census Income), using both GPT-4 and Gemini-Pro as encoders and GPT-4 as the decoder.

Experimental results

Research questions

• RQ1 Can emoji-based prompt obfuscation effectively hide sensitive user data from both humans and the LLM provider while preserving the model’s ability to generate accurate responses?
• RQ2 Does encrypting prompts using emojis maintain or improve task performance compared to unencrypted prompts in LLM applications?
• RQ3 To what extent can original sensitive content be reconstructed from emoji-encrypted prompts, indicating the strength of the obfuscation?
• RQ4 How does the performance of the encryption-decryption pipeline vary across different LLMs (e.g., GPT-4 vs. Gemini-Pro) and data types (reusable vs. non-reusable text)?
• RQ5 Can the method be applied universally across diverse NLP tasks without requiring model fine-tuning or architectural changes?

Key findings

• EmojiCrypt achieved a cosine similarity of 0.567 between original and reconstructed product titles on the Amazon Beauty dataset when using GPT-4 for both encryption and decryption, significantly below the optimal safety threshold of 0.307, indicating strong obfuscation.
• On the IMDB review dataset, the similarity score was 0.755 for GPT-4 + GPT-4 and 0.607 for Gemini + GPT-4, both exceeding the optimal safety threshold of 0.411, confirming that reconstructed text remains semantically distinct from the original.
• For the Census Income dataset, the similarity score was 0.461 (GPT-4 + GPT-4) and 0.467 (Gemini + GPT-4), both above the optimal safety threshold of 0.295, demonstrating effective data masking.
• In all three tasks—personalized recommendation, sentiment analysis, and tabular data analysis—EmojiCrypt maintained or even improved task accuracy compared to unencrypted prompts, with no need for model fine-tuning.
• The method effectively prevents reconstruction of original sensitive content, as the average similarity between encrypted prompts and random samples from the same dataset remained below the safety threshold, confirming robustness.
• The results show that LLMs can process emoji-encrypted prompts with high fidelity, indicating that semantic information is preserved in the obfuscation process despite the apparent randomness of the emoji representation.