[Paper Review] Fast Private Data Release Algorithms for Sparse Queries

Avrim Blum, Aaron Roth|arXiv (Cornell University)|Nov 29, 2011

Privacy-Preserving Technologies in Data7 citations

TL;DR

This paper presents efficient, differentially private algorithms for releasing answers to sparse statistical queries—queries that depend on only polynomially many elements of the data universe. Using random projections and noise injection, the authors achieve accuracy independent of the universe size |X|, with running times polynomial in the database size n and query sparsity m, enabling practical deployment even in infinite-universe settings.

ABSTRACT

We revisit the problem of accurately answering large classes of statistical queries while preserving differential privacy. Previous approaches to this problem have either been very general but have not had run-time polynomial in the size of the database, have applied only to very limited classes of queries, or have relaxed the notion of worst-case error guarantees. In this paper we consider the large class of <em>sparse</em> queries, which take non-zero values on only polynomially many universe elements. We give efficient query release algorithms for this class, in both the interactive and the non-interactive setting. Our algorithms also achieve better accuracy bounds than previous general techniques do when applied to sparse queries: our bounds are independent of the universe size. In fact, even the runtime of our interactive mechanism is independent of the universe size, and so can be implemented in the “infinite universe” model in which no finite universe need be specified by the data curator.

Motivation & Objective

• To develop efficient, differentially private algorithms for answering large classes of sparse statistical queries.
• To overcome the runtime and accuracy limitations of prior general-purpose differentially private mechanisms that scale with the data universe size |X|.
• To enable practical private data analysis for specialists who possess domain knowledge about the data, by focusing on sparse queries that are meaningful in context.
• To achieve both computational efficiency and strong accuracy guarantees in both interactive and non-interactive query release settings.

Proposed method

• Employs random projection matrices with r-wise independent entries to compress the database into a lower-dimensional representation.
• Uses Laplace noise added to the projected data to ensure (ε, δ)-differential privacy.
• Applies Johnson-Lindenstrauss-type concentration bounds to control distortion from random projection.
• Uses tail bounds on sums of Laplace and Rademacher random variables to bound privacy-induced error.
• Constructs a non-interactive data structure that encodes answers to all queries in the class, enabling fast evaluation.
• Leverages the sparsity of queries (m-sparse) to decouple accuracy from |X|, achieving bounds that depend only on m, n, and k.

Experimental results

Research questions

• RQ1 Can we design differentially private query release algorithms that are efficient (polynomial time) and accurate for large classes of sparse queries?
• RQ2 Can we achieve accuracy independent of the data universe size |X|, especially when |X| is exponentially large?
• RQ3 Is it possible to design a non-interactive mechanism that supports super-polynomially many queries with non-trivial accuracy and polynomial runtime?
• RQ4 Can the algorithms work in the infinite universe model, where no finite upper bound on attribute values is assumed?
• RQ5 Can random projections be used effectively to reduce dimensionality while preserving privacy and accuracy for sparse queries?

Key findings

• The interactive mechanism achieves α-accuracy with runtime per query of Õ(m/α²), independent of |X|, enabling use in infinite-universe settings.
• The non-interactive mechanism runs in time polynomial in n, m, and log|X|, and achieves α-accuracy bounded by Õ(√(log k / (m log(1/δ)) / (εn)))).
• Accuracy bounds are independent of |X|, offering significant improvement over general mechanisms whose error scales with log|X|.
• The non-interactive mechanism supports super-polynomially many conjunctions (e.g., Cd−log n), a class of size dk, with non-trivial accuracy and polynomial time.
• The algorithms are robust to adaptive query selection and maintain privacy under composition.
• The framework enables private release of data useful to domain experts, even when queries are sparse and the data universe is large or infinite.