[Paper Review] Optimal Algorithms for Mean Estimation under Local Differential Privacy

Hilal Asi, Vitaly Feldman+1 more|arXiv (Cornell University)|May 5, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper establishes that PrivUnit with optimized parameters achieves the minimum possible variance for mean estimation under local differential privacy, proving it is optimal among a broad class of non-interactive, unbiased protocols. It further introduces PrivUnitG—a Gaussian-based variant—that enables precise analytical computation of optimal error constants and confirms the asymptotic optimal constant $ C^* \approx 0.614 $ as $ \varepsilon \to \infty $. The results are derived via symmetrization and linear programming techniques on local randomizers.

ABSTRACT

We study the problem of mean estimation of $\ell_2$-bounded vectors under the constraint of local differential privacy. While the literature has a variety of algorithms that achieve the asymptotically optimal rates for this problem, the performance of these algorithms in practice can vary significantly due to varying (and often large) hidden constants. In this work, we investigate the question of designing the protocol with the smallest variance. We show that PrivUnit (Bhowmick et al. 2018) with optimized parameters achieves the optimal variance among a large family of locally private randomizers. To prove this result, we establish some properties of local randomizers, and use symmetrization arguments that allow us to write the optimal randomizer as the optimizer of a certain linear program. These structural results, which should extend to other problems, then allow us to show that the optimal randomizer belongs to the PrivUnit family. We also develop a new variant of PrivUnit based on the Gaussian distribution which is more amenable to mathematical analysis and enjoys the same optimality guarantees. This allows us to establish several useful properties on the exact constants of the optimal error as well as to numerically estimate these constants.

Motivation & Objective

• To identify the protocol with the smallest variance for mean estimation under local differential privacy, particularly focusing on minimizing hidden constants in asymptotic error rates.
• To characterize the structure of optimal local randomizers through symmetrization and linear programming, enabling exact optimality proofs.
• To develop a new algorithm, PrivUnitG, based on the Gaussian distribution that preserves optimality guarantees while allowing precise mathematical analysis of error constants.
• To numerically and analytically estimate the exact constants in the optimal error, especially in high-dimensional settings.
• To show that the optimal error constant converges to $ C^* \approx 0.614 $ as $ \varepsilon \to \infty $, providing a benchmark for practical implementations.

Proposed method

• The authors define a class of non-interactive, unbiased protocols and analyze their worst-case mean squared error, using symmetrization to reduce the optimization problem to a linear program.
• They prove that the optimal randomizer must satisfy specific symmetry and invariance properties, which allows expressing the problem as a constrained optimization over probability distributions.
• By analyzing the structure of the optimal randomizer, they show it belongs to the PrivUnit family, thus proving PrivUnit with optimized parameters is optimal within this class.
• They introduce PrivUnitG, a variant using the Gaussian distribution, which simplifies analysis and allows exact computation of error constants due to its dimension-independent parameterization.
• They derive bounds on the ratio of error constants between PrivUnitG and PrivUnit, showing they are asymptotically equivalent as $ d \to \infty $.
• They use concentration inequalities and tail probability bounds on Gaussian and spherical random variables to compare the performance of PrivUnit and PrivUnitG, establishing $ \sqrt{\mathsf{Err}_G / \mathsf{Err}_2} \leq 1 + O\left(\sqrt{\frac{\varepsilon + \log d}{d}}\right) $.

Experimental results

Research questions

• RQ1 What is the optimal local randomizer for mean estimation under $ \varepsilon $-local differential privacy, and can it be characterized explicitly?
• RQ2 Does PrivUnit with optimized parameters achieve the minimal possible variance among all non-interactive, unbiased protocols in the local privacy model?
• RQ3 Can a variant of PrivUnit based on the Gaussian distribution achieve the same asymptotic error as PrivUnit while enabling exact analytical computation of error constants?
• RQ4 How do the optimal error constants behave as the privacy parameter $ \varepsilon \to \infty $ and dimension $ d \to \infty $?
• RQ5 What is the limiting value of the optimal error constant $ C^* $, and how quickly does the constant $ C_{\varepsilon,d} $ converge to it?

Key findings

• PrivUnit with optimized parameters achieves the optimal variance among a large family of locally private randomizers for mean estimation under $ \varepsilon $-local differential privacy.
• The optimal randomizer belongs to the PrivUnit family, as proven via symmetrization and linear programming reformulation of the optimization problem.
• PrivUnitG, a Gaussian-based variant of PrivUnit, achieves the same asymptotic error as PrivUnit up to a $ (1+o(1)) $ factor as $ d \to \infty $, enabling precise error constant analysis.
• The optimal error constant $ C_{\varepsilon,d} $ converges to $ C_{\varepsilon} $ as $ d \to \infty $, with convergence rate $ O\left(\frac{\varepsilon + \log d}{d}\right) $.
• The limiting constant $ C^* = \lim_{\varepsilon \to \infty} C_{\varepsilon} $ exists and is approximately $ 0.614 $, providing a sharp benchmark for the best possible error in high-privacy regimes.
• The ratio of error constants between PrivUnitG and PrivUnit is bounded by $ 1 + O\left(\sqrt{\frac{\varepsilon + \log d}{d}}\right) $, confirming their asymptotic equivalence.