[Paper Review] Efficient Differentially Private $F_0$ Linear Sketching

Rasmus Pagh, Nina Mesing Stausholm|IT University Of Copenhagen (IT University of Copenhagen)|Jan 31, 2020

Privacy-Preserving Technologies in Data44 references4 citations

TL;DR

This paper presents an efficient differentially private linear sketch for estimating the weighted $F_0$ norm, defined as $\|x \circ w\|_1$, where $x$ is a binary vector and $w$ is a weight vector in $(0,1]^u$. By combining a GF(2)-linear sketch with randomized response noise, the method achieves $\varepsilon$-differential privacy, $O(\log^2 u / \varepsilon^2 \beta^2)$ sketch size, and $(1\pm\beta)$ relative error with additive error $O(\log u / \varepsilon^2 \beta^2)$, while supporting efficient merging of sketches for distributed streaming applications.

ABSTRACT

A powerful feature of linear sketches is that from sketches of two data vectors, one can compute the sketch of the difference between the vectors. This allows us to answer fine-grained questions about the difference between two data sets. In this work, we consider how to construct sketches for weighted $F_0$, i.e., the summed weights of the elements in the data set, that are small, differentially private, and computationally efficient. Let a weight vector $w\in(0,1]^u$ be given. For $x\in\{0,1\}^u$ we are interested in estimating $\Vert x\circ w\Vert_1$ where $\circ$ is the Hadamard product (entrywise product). Building on a technique of Kushilevitz et al.~(STOC 1998), we introduce a sketch (depending on $w$) that is linear over GF(2), mapping a vector $x\in \{0,1\}^u$ to $Hx\in\{0,1\}^τ$ for a matrix $H$ sampled from a suitable distribution $\mathcal{H}$. Differential privacy is achieved by using randomized response, flipping each bit of $Hx$ with probability $p<1/2$. We show that for every choice of $0

Motivation & Objective

• To design a differentially private linear sketch that supports efficient, accurate estimation of the weighted $F_0$ norm $\|x \circ w\|_1$ in a streaming or distributed setting.
• To overcome the computational inefficiency of prior differentially private $F_0$ sketching methods that required quasipolynomial or superlinear time for noise calibration.
• To extend existing differentially private sketching techniques to handle weighted data, where each element has a weight in $(0,1]$.
• To enable secure and private estimation of set operations such as symmetric difference and union size in distributed data streams.
• To ensure that sketches can be merged efficiently while preserving differential privacy and estimation accuracy.

Proposed method

• The method uses a linear sketch over GF(2) defined by a matrix $H$ sampled from a distribution $\mathcal{H}$, mapping input $x \in \{0,1\}^u$ to $Hx \in \{0,1\}^\tau$.
• Differential privacy is achieved by adding independent Bernoulli noise $\varphi_j \sim \text{Bernoulli}(p)$ to each entry of $Hx$, resulting in the noisy sketch $Hx + \varphi$ over GF(2).
• The sketch size $\tau$ is set to $O(\log^2 u \cdot \varepsilon^{-2} \beta^{-2})$ to ensure high-probability accuracy within $1\pm\beta$ relative error and $O(\log u \cdot \varepsilon^{-2} \beta^{-2})$ additive error.
• The estimation algorithm computes the number of ones in each row of $Hx + \varphi$, uses interval estimation across $\log u$ levels, and computes the intersection of intervals to obtain a final estimate.
• For distributed streaming, a pre-sampling layer is applied: each item is independently sampled with probability $1/2$ before sketching, ensuring that the sketch remains robust to duplicates and supports union estimation.
• The method leverages Chernoff bounds to ensure that the fraction of items sampled an odd number of times is close to $1/2$, enabling a two-fold scaling of the estimate to recover the true weighted count.

Experimental results

Research questions

• RQ1 Can we design a differentially private linear sketch for weighted $F_0$ estimation that is both computationally efficient and space-efficient?
• RQ2 How can we achieve $\varepsilon$-differential privacy using randomized response on a GF(2)-linear sketch without incurring quasipolynomial or superlinear time costs?
• RQ3 What sketch size is required to achieve $1\pm\beta$ relative error with high probability, and how does it compare to known lower bounds?
• RQ4 Can the sketch be efficiently merged in a distributed streaming setting to estimate the union or symmetric difference of two data sets?
• RQ5 Does the method improve upon prior work in terms of error bounds when $\beta = o(1/\sqrt{\bar{m}})$ and $\log u / \varepsilon = \bar{m}^{o(1)}$?

Key findings

• The sketch size is $\tau = O(\log^2 u \cdot \varepsilon^{-2} \beta^{-2})$, which is polynomially related to the known lower bound of $\Omega(\log u \cdot \beta^{-2})$ for $F_0$ estimation.
• The method achieves $\varepsilon$-differential privacy over the randomness in the noise vector $\varphi$, with no dependence on the data or the sketch matrix $H$.
• The estimation error is bounded by a relative error of $1\pm\beta$ with probability $1 - u^{-1}$, and an additive error of $O(\log u \cdot \varepsilon^{-2} \beta^{-2})$.
• The algorithm runs in time $O(\tau)$ for estimation and $O(\|x\|_0 \log u)$ for sketching, making it computationally efficient for large-scale data.
• The method supports merging of sketches: $H(x_1 + x_2) + (\varphi_1 + \varphi_2)$ yields a valid sketch for the symmetric difference, enabling private union size estimation in distributed streams.
• In the distributed streaming model, the pre-sampling technique ensures that the sketch of the union of two streams is equivalent to the sketch of a single stream with independently sampled items, preserving privacy and accuracy.