[Paper Review] Gaussian Differential Privacy

Jinshuo Dong, Aaron Roth+1 more|arXiv (Cornell University)|May 7, 2019

Privacy-Preserving Technologies in Data40 references102 citations

TL;DR

Introduces f-differential privacy (f-DP) as a hypothesis-testing based relaxation of DP, with Gaussian differential privacy (GDP) as a canonical single-parameter family, and develops composition theorems, subsampling amplification, and applications to SGD.

ABSTRACT

Differential privacy has seen remarkable success as a rigorous and practical formalization of data privacy in the past decade. This privacy definition and its divergence based relaxations, however, have several acknowledged weaknesses, either in handling composition of private algorithms or in analyzing important primitives like privacy amplification by subsampling. Inspired by the hypothesis testing formulation of privacy, this paper proposes a new relaxation, which we term `$f$-differential privacy' ($f$-DP). This notion of privacy has a number of appealing properties and, in particular, avoids difficulties associated with divergence based relaxations. First, $f$-DP preserves the hypothesis testing interpretation. In addition, $f$-DP allows for lossless reasoning about composition in an algebraic fashion. Moreover, we provide a powerful technique to import existing results proven for original DP to $f$-DP and, as an application, obtain a simple subsampling theorem for $f$-DP. In addition to the above findings, we introduce a canonical single-parameter family of privacy notions within the $f$-DP class that is referred to as `Gaussian differential privacy' (GDP), defined based on testing two shifted Gaussians. GDP is focal among the $f$-DP class because of a central limit theorem we prove. More precisely, the privacy guarantees of \emph{any} hypothesis testing based definition of privacy (including original DP) converges to GDP in the limit under composition. The CLT also yields a computationally inexpensive tool for analyzing the exact composition of private algorithms. Taken together, this collection of attractive properties render $f$-DP a mathematically coherent, analytically tractable, and versatile framework for private data analysis. Finally, we demonstrate the use of the tools we develop by giving an improved privacy analysis of noisy stochastic gradient descent.

Motivation & Objective

• Motivate the need for a hypothesis-testing based privacy relaxation to address composition and amplification limitations of classical DP.
• Define f-differential privacy (f-DP) via trade-off functions that capture the optimal type I/type II error trade-off under dataset neighboring.
• Introduce Gaussian differential privacy (GDP) as the canonical single-parameter instantiation within f-DP.
• Develop composition theorems and a central limit theorem style result showing convergence to GDP under composition.
• Demonstrate practical utility by improving privacy analyses for stochastic gradient descent and relating f-DP to existing DP frameworks.

Proposed method

• Formalize f-DP using trade-off functions T(P,Q) and define symmetric trade-off functions.
• Specialize to GDP by characterizing G_mu via Gaussian hypothesis testing between N(0,1) and N(mu,1).
• Show that GDP is preserved under post-processing and provide a primal-dual view linking f-DP to (epsilon,delta)-DP guarantees.
• Prove a composition theorem and a Berry-Esseen type CLT for composition to approximate cumulative privacy loss.
• Establish the Gaussian mechanism as mu-GDP and derive exact GDP bounds for Gaussian noise addition.
• Apply the framework to analyze the privacy of noisy stochastic gradient descent (SGD) and compare to prior DP analyses.

Experimental results

Research questions

• RQ1 How can we formulate differential privacy as a functional trade-off problem to better handle composition and subsampling?
• RQ2 What is a canonical single-parameter privacy notion within f-DP that aligns with hypothesis testing interpretations?
• RQ3 How does GDP relate to the Gaussian mechanism, and can we characterize composition limits via a central limit theorem?
• RQ4 Can f-DP be converted to and from traditional (epsilon, delta)-DP guarantees to leverage existing results?
• RQ5 How does GDP enable sharper privacy analyses for SGD compared to existing DP methods?

Key findings

• f-DP provides a tight, composition-friendly privacy framework based on trade-off functions.
• GDP precisely characterizes the Gaussian mechanism and serves as a canonical privacy guarantee under composition limits.
• GDP converges to under-composed privacy loss via a central limit theorem for hypothesis-testing based privacy notions.
• There exists a primal-dual correspondence between f-DP and families of (epsilon, delta)-DP guarantees enabling transfer of results.
• Post-processing does not degrade f-DP guarantees, and Blackwell-type results show f-DP is maximally informative among certain privacy notions.
• The framework yields sharper SGD privacy analyses than prior divergence-based methods, via GDP-based composition approximations.