[Paper Review] Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds

Mark Bun, Thomas Steinke|arXiv (Cornell University)|May 6, 2016

Privacy-Preserving Technologies in Data8 references65 citations

TL;DR

This paper introduces zero-concentrated differential privacy (zCDP), a reformulation of concentrated differential privacy using Rényi divergence to better capture subgaussian privacy loss. It provides tighter composition bounds, unifies with approximate differential privacy, and establishes improved lower bounds and mechanisms for privacy-preserving computation.

ABSTRACT

"Concentrated differential privacy" was recently introduced by Dwork and Rothblum as a relaxation of differential privacy, which permits sharper analyses of many privacy-preserving computations. We present an alternative formulation of the concept of concentrated differential privacy in terms of the Renyi divergence between the distributions obtained by running an algorithm on neighboring inputs. With this reformulation in hand, we prove sharper quantitative results, establish lower bounds, and raise a few new questions. We also unify this approach with approximate differential privacy by giving an appropriate definition of "approximate concentrated differential privacy."

Motivation & Objective

• Address the limitations of approximate differential privacy in composition analysis, particularly the unwieldy and non-tight bounds in advanced composition theorems.
• Provide a cleaner, more analytically tractable formulation of concentrated differential privacy using Rényi divergence.
• Unify zCDP with approximate differential privacy to allow composition of mechanisms under different privacy guarantees.
• Establish tighter lower bounds on privacy loss for zCDP and derive improved mechanisms for practical use.
• Enable sharper quantitative analysis of privacy-preserving algorithms, especially for Gaussian mechanisms and repeated queries.

Proposed method

• Reformulate concentrated differential privacy using Rényi divergence between output distributions on neighboring datasets, defining zero-concentrated differential privacy (zCDP).
• Prove that zCDP implies differential privacy with explicit bounds on privacy parameters, enabling tighter composition analysis.
• Derive the Gaussian mechanism under zCDP, showing it achieves optimal privacy-utility trade-offs for noise addition.
• Establish composition and postprocessing properties for zCDP, showing smooth degradation under sequential computation.
• Introduce approximate zCDP as a unifying framework that generalizes both pure and approximate differential privacy.
• Use concentration inequalities and tail bounds to derive explicit privacy loss bounds, particularly leveraging subgaussianity of privacy loss.

Experimental results

Research questions

• RQ1 Can concentrated differential privacy be reformulated using Rényi divergence to yield tighter and more analytically tractable privacy guarantees?
• RQ2 How does zCDP compare quantitatively to pure and approximate differential privacy in terms of composition and utility?
• RQ3 What are the tightest possible lower bounds on privacy loss under zCDP, and how do they constrain mechanism design?
• RQ4 Can zCDP be extended to a unified framework that includes approximate differential privacy?
• RQ5 What improvements does zCDP offer in the analysis of the Gaussian mechanism and repeated query workloads?

Key findings

• Zero-concentrated differential privacy (zCDP) is formally defined via Rényi divergence, providing a cleaner and more analytically powerful alternative to mean-concentrated differential privacy.
• The Gaussian mechanism under zCDP achieves optimal privacy-utility trade-offs, with tight bounds on privacy loss that improve upon the advanced composition theorem.
• zCDP enables tighter composition bounds: for k mechanisms with zCDP parameter ρ, the composition satisfies (ε, δ)-DP with δ decaying as exp(−(ε−ρ)²/(4ρ)) for ε ≥ ρ.
• The paper derives a unified composition theorem that subsumes both pure and approximate differential privacy, allowing mixed composition of zCDP and (ε,δ)-DP mechanisms.
• A new lower bound on privacy loss is established, showing that zCDP cannot be significantly improved in terms of composition efficiency.
• The results improve the advanced composition theorem by saving a constant factor in the privacy parameter and reducing the dependence on logarithmic terms in δ.