[Paper Review] Text Embeddings Reveal (Almost) As Much As Text

John X. Morris, Volodymyr Kuleshov+2 more|arXiv (Cornell University)|Oct 10, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes Vec2Text, a multi-step iterative method that recovers full text from dense text embeddings by refining hypotheses through repeated embedding and correction. It achieves 92% exact recovery of 32-token inputs and extracts 89% of full names from clinical notes, demonstrating that embeddings leak nearly as much private information as raw text.

ABSTRACT

How much private information do text embeddings reveal about the original text? We investigate the problem of embedding extit{inversion}, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a naïve model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover $92\%$ of $32 ext{-token}$ text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. Our code is available on Github: \href{https://github.com/jxmorris12/vec2text}{github.com/jxmorris12/vec2text}.

Motivation & Objective

• Investigate whether dense text embeddings leak sufficient information to reconstruct the original input text.
• Address the privacy threat posed by third-party vector databases storing embeddings instead of raw text.
• Develop a practical method to invert state-of-the-art text embeddings without access to model weights.
• Demonstrate that embeddings from real-world retrieval models can be inverted to recover sensitive information like names.
• Establish that embeddings should be treated as sensitive as raw text due to equivalent privacy leakage.

Proposed method

• Frame embedding inversion as a controlled generation problem: generate text whose embedding is close to a target embedding.
• Use a conditional language model trained to reconstruct input text from its embedding, learning a distribution p(x|e;θ) via maximum likelihood.
• Apply iterative refinement: generate a hypothesis text, embed it, compute the cosine similarity between the hypothesis embedding and the target, and correct the text based on the difference.
• Use a beam search with a corrective module to iteratively improve the hypothesis over multiple rounds.
• Leverage black-box access to the embedding model to query the embedding of each hypothesis and guide refinement.
• Train the model on embeddings from two state-of-the-art encoders and evaluate generalization on unseen domains and clinical data.

Experimental results

Research questions

• RQ1 To what extent can full-length, ordered text sequences be reconstructed from their dense embeddings?
• RQ2 Can a black-box embedding model be inverted to recover exact input text without access to gradients or model weights?
• RQ3 How effective is iterative correction in recovering text from embeddings compared to direct generation?
• RQ4 What is the privacy risk of storing embeddings instead of raw text in vector databases?
• RQ5 Can sensitive personal information like full names be extracted from embeddings of clinical notes?

Key findings

• Vec2Text recovers 92% of 32-token text inputs exactly using iterative refinement and black-box embedding queries.
• The method achieves a BLEU score of 97.3 on 32-token inputs, indicating near-perfect reconstruction quality.
• The model generalizes to unseen domains from the BEIR benchmark, perfectly recovering inputs across diverse text types.
• In clinical note embeddings from MIMIC, 89% of full names are successfully recovered, highlighting risks in healthcare data.
• The results show that text embeddings reveal nearly as much private information as the original text, implying equivalent privacy protections are needed.
• The study demonstrates that embeddings should be treated as sensitive personal data, comparable to raw text, due to equivalent privacy leakage.