[Paper Review] Reviewing and Improving the Gaussian Mechanism for Differential Privacy

Jun Zhao, Teng Wang+8 more|arXiv (Cornell University)|Nov 27, 2019

Privacy-Preserving Technologies in DataComputer Science50 references19 citations

TL;DR

This paper identifies and corrects a long-standing flaw in the classical Gaussian mechanism for $(\epsilon,\delta)$-differential privacy, showing that the widely used noise bounds from Dwork et al. (2006) and Dwork & Roth (2014) fail to guarantee privacy for large $\epsilon$. The authors derive new closed-form upper bounds for the optimal noise parameter $\sigma_{\texttt{DP-OPT}}$, ensuring $(\epsilon,\delta)$-DP for all $\epsilon > 0$ while improving utility over classical mechanisms.

ABSTRACT

Differential privacy provides a rigorous framework to quantify data privacy, and has received considerable interest recently. A randomized mechanism satisfying $(ε, δ)$-differential privacy (DP) roughly means that, except with a small probability $δ$, altering a record in a dataset cannot change the probability that an output is seen by more than a multiplicative factor $e^ε $. A well-known solution to $(ε, δ)$-DP is the Gaussian mechanism initiated by Dwork et al. [1] in 2006 with an improvement by Dwork and Roth [2] in 2014, where a Gaussian noise amount $\sqrt{2\ln \frac{2}δ} imes \fracΔε$ of [1] or $\sqrt{2\ln \frac{1.25}δ} imes \fracΔε$ of [2] is added independently to each dimension of the query result, for a query with $\ell_2$-sensitivity $Δ$. Although both classical Gaussian mechanisms [1,2] assume $0 < ε\leq 1$, our review finds that many studies in the literature have used the classical Gaussian mechanisms under values of $ε$ and $δ$ where the added noise amounts of [1,2] do not achieve $(ε,δ)$-DP. We obtain such result by analyzing the optimal noise amount $σ_{DP-OPT}$ for $(ε,δ)$-DP and identifying $ε$ and $δ$ where the noise amounts of classical mechanisms are even less than $σ_{DP-OPT}$. Since $σ_{DP-OPT}$ has no closed-form expression and needs to be approximated in an iterative manner, we propose Gaussian mechanisms by deriving closed-form upper bounds for $σ_{DP-OPT}$. Our mechanisms achieve $(ε,δ)$-DP for any $ε$, while the classical mechanisms [1,2] do not achieve $(ε,δ)$-DP for large $ε$ given $δ$. Moreover, the utilities of our mechanisms improve those of [1,2] and are close to that of the optimal yet more computationally expensive Gaussian mechanism.

Motivation & Objective

• To identify and correct the widespread misuse of the classical Gaussian mechanisms (Dwork-2006 and Dwork-2014) in the literature, which fail to guarantee $(\epsilon,\delta)$-differential privacy for large $\epsilon$.
• To formally analyze the optimal noise parameter $\sigma_{\texttt{DP-OPT}}$ for $(\epsilon,\delta)$-DP and show that classical mechanisms use noise levels below the required threshold for large $\epsilon$.
• To derive new closed-form upper bounds for $\sigma_{\texttt{DP-OPT}}$ that are valid for all $\epsilon > 0$, enabling correct and efficient implementation of the Gaussian mechanism.
• To improve the utility of the Gaussian mechanism by reducing noise while maintaining privacy guarantees, approaching the performance of the optimal but computationally expensive $\sigma_{\texttt{DP-OPT}}$.
• To provide a comprehensive analysis of $(\epsilon,\delta)$-probabilistic differential privacy (pDP), including its relationship to standard DP and noise requirements.

Proposed method

• The authors derive the optimal noise parameter $\sigma_{\texttt{DP-OPT}}$ for $(\epsilon,\delta)$-DP by solving the privacy constraint using the cumulative distribution function of the Gaussian distribution and the complementary error function.
• They prove that for large $\epsilon$, $\sigma_{\texttt{DP-OPT}} = \Theta(1/\sqrt{\epsilon})$, contradicting the classical $\Theta(1/\epsilon)$ scaling, which invalidates the classical mechanisms for large $\epsilon$.
• The paper proposes new closed-form upper bounds for $\sigma_{\texttt{DP-OPT}}$ that are tighter than classical bounds and valid for all $\epsilon > 0$, derived via analytical inequalities on the inverse complementary error function.
• The authors validate their bounds using rigorous mathematical proofs, including Craig’s formula for the complementary error function and bounds on the inverse function $\operatorname{inverfc}(y)$.
• They compare the proposed mechanisms to classical ones and the optimal $\sigma_{\texttt{DP-OPT}}$, demonstrating improved utility through theoretical and numerical analysis.
• An extensive discussion of $(\epsilon,\delta)$-probabilistic differential privacy (pDP) is included, clarifying its relationship to standard DP and deriving corresponding noise requirements.

Experimental results

Research questions

• RQ1 Why do classical Gaussian mechanisms (Dwork-2006 and Dwork-2014) fail to achieve $(\epsilon,\delta)$-DP for large $\epsilon$ values, despite their widespread use?
• RQ2 What is the correct asymptotic scaling of the optimal noise parameter $\sigma_{\texttt{DP-OPT}}$ for large $\epsilon$, and how does it differ from the classical $\Theta(1/\epsilon)$ assumption?
• RQ3 Can closed-form upper bounds for $\sigma_{\texttt{DP-OPT}}$ be derived that are valid for all $\epsilon > 0$ and ensure $(\epsilon,\delta)$-DP?
• RQ4 How do the proposed mechanisms compare in utility to the classical mechanisms and the optimal $\sigma_{\texttt{DP-OPT}}$?
• RQ5 What is the relationship between $(\epsilon,\delta)$-DP and $(\epsilon,\delta)$-probabilistic differential privacy (pDP), and how do noise requirements differ?

Key findings

• The classical Gaussian mechanisms (Dwork-2006 and Dwork-2014) do not guarantee $(\epsilon,\delta)$-DP for large $\epsilon$ because their noise levels are below the required $\sigma_{\texttt{DP-OPT}}$ for such $\epsilon$ values.
• For large $\epsilon$, the optimal noise parameter scales as $\Theta(1/\sqrt{\epsilon})$, not $\Theta(1/\epsilon)$, invalidating the classical assumptions.
• The proposed closed-form upper bounds for $\sigma_{\texttt{DP-OPT}}$ ensure $(\epsilon,\delta)$-DP for all $\epsilon > 0$ and are tighter than classical bounds, improving utility.
• The new mechanisms achieve significantly better utility than the classical mechanisms while remaining close to the optimal $\sigma_{\texttt{DP-OPT}}$, which requires iterative computation.
• The paper formally proves that $\sigma_{\texttt{Dwork-2006}} = \sqrt{2\ln(2/\delta)} \cdot \Delta / \epsilon$ and $\sigma_{\texttt{Dwork-2014}} = \sqrt{2\ln(1.25/\delta)} \cdot \Delta / \epsilon$ are insufficient for large $\epsilon$, even when $\delta$ is small.
• The authors derive and prove a new upper bound: $\operatorname{inverfc}(y) < \sqrt{\ln\left(\frac{2}{\sqrt{8y+1}-1}\right)}$, which is used to construct the new closed-form noise bounds.