[Paper Review] Differentially Private Linear Sketches: Efficient Implementations and Applications

Fuheng Zhao, Dan Qiao+4 more|arXiv (Cornell University)|May 19, 2022

Privacy-Preserving Technologies in Data11 citations

TL;DR

This paper proposes differentially private linear sketches by adding Gaussian noise at initialization, enabling privacy-preserving frequency estimation, top-K item identification, and quantile approximation in the turnstile model. The approach maintains high utility with negligible error growth, achieves concentrated differential privacy, and introduces the first differentially private quantile sketch in the turnstile model with provable error bounds.

ABSTRACT

Linear sketches have been widely adopted to process fast data streams, and they can be used to accurately answer frequency estimation, approximate top K items, and summarize data distributions. When data are sensitive, it is desirable to provide privacy guarantees for linear sketches to preserve private information while delivering useful results with theoretical bounds. We show that linear sketches can ensure privacy and maintain their unique properties with a small amount of noise added at initialization. From the differentially private linear sketches, we showcase that the state-of-the-art quantile sketch in the turnstile model can also be private and maintain high performance. Experiments further demonstrate that our proposed differentially private sketches are quantitatively and qualitatively similar to noise-free sketches with high utilization on synthetic and real datasets.

Motivation & Objective

• To enable differential privacy in linear sketches while preserving their core properties such as efficiency and accuracy.
• To support both insertions and deletions in the turnstile model, unlike prior works restricted to insertion-only or bounded-deletion models.
• To develop the first differentially private quantile sketch in the turnstile model with provable error bounds.
• To ensure post-processing immunity by maintaining privacy across arbitrary query operations.
• To demonstrate that private sketches achieve utility close to noise-free counterparts on both synthetic and real-world datasets.

Proposed method

• Add Gaussian noise to the initial sketch matrix at initialization to achieve concentrated differential privacy, without modifying update or query procedures.
• Leverage the Gaussian mechanism to provide strong privacy guarantees while preserving the original sketch structure and properties.
• Use the private CountSketch as a foundation to construct a differentially private quantile sketch (DP DCS) for the turnstile model.
• Derive a uniform error bound for the supremum of estimation error across all items, ensuring robustness for all queries.
• Utilize ideal random hash functions (replacing with cryptographic ones in practice) to ensure randomness and privacy.
• Apply post-processing immunity to allow arbitrary number of queries without degrading privacy guarantees.

Experimental results

Research questions

• RQ1 Can linear sketches be made differentially private with minimal impact on utility by adding noise only at initialization?
• RQ2 Can the proposed method support both insertions and deletions in the turnstile model while maintaining privacy and accuracy?
• RQ3 Is it possible to construct the first differentially private quantile sketch in the turnstile model using private linear sketches?
• RQ4 How does the error of the private sketches scale with database size, privacy budget, and universe size?
• RQ5 Can the private sketches maintain the no-underestimation property of Count-Min while ensuring differential privacy?

Key findings

• The proposed differentially private linear sketches maintain high utility, with average rank error one order of magnitude below the theoretical upper bound of γ·N = 10³ for N = 10⁵.
• For DP DCS, increasing the number of evenly spaced quantiles does not increase average rank error, indicating stable performance across quantile resolution.
• The average rank error increases with decreasing privacy budget (e.g., ρ = 0.1), but the increase remains small relative to database size, especially for large N.
• The private Count-Min sketch preserves the no-underestimation property with high probability, unlike Laplace-based methods that break this property.
• The uniform error bound derived for supₓ|f̂(x)−f̃(x)| enables tighter overall error control and can be combined with point-wise bounds via triangle inequality.
• The method achieves concentrated differential privacy via the Gaussian mechanism, offering stronger guarantees than approximate DP from binomial noise, and outperforms Laplace-based approaches in utility and property preservation.