[Paper Review] Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences

Borja Balle, Gilles Barthe+1 more|arXiv (Cornell University)|Jul 4, 2018

Privacy-Preserving Technologies in Data2 references105 citations

TL;DR

The paper presents a unified framework using α-divergences, couplings, and privacy profiles to derive tight privacy amplification bounds for various subsampling schemes and neighboring relations, and proves tightness via lower bounds.

ABSTRACT

Differential privacy comes equipped with multiple analytical tools for the design of private data analyses. One important tool is the so-called "privacy amplification by subsampling" principle, which ensures that a differentially private mechanism run on a random subsample of a population provides higher privacy guarantees than when run on the entire population. Several instances of this principle have been studied for different random subsampling methods, each with an ad-hoc analysis. In this paper we present a general method that recovers and improves prior analyses, yields lower bounds and derives new instances of privacy amplification by subsampling. Our method leverages a characterization of differential privacy as a divergence which emerged in the program verification community. Furthermore, it introduces new tools, including advanced joint convexity and privacy profiles, which might be of independent interest.

Motivation & Objective

• Develop a general, tight framework to analyze privacy amplification by subsampling across different subsampling schemes and neighbor relations.
• Unify existing results and derive new amplification bounds using a divergence-based characterization of DP.
• Introduce advanced joint convexity and privacy profiles to bound mixtures arising from subsampling.
• Establish tightness of bounds via lower bounds and coupling-based proofs.

Proposed method

• Model differential privacy via α-divergences and privacy profiles (D_α) to capture tight DP guarantees.
• Use advanced joint convexity to bound D_{e^{ε′}} between overlapping mixtures arising from subsampling.
• Apply maximal coupling to relate subsampled distributions and derive η-dependent amplification ε′ via ε′ = log(1 + η(e^{ε}-1)).
• Bound right-hand side divergences with group-privacy profiles and coupling-based decompositions.
• Provide a generic framework that recovers existing results and yields new, tight bounds for various subsampling schemes.

Experimental results

Research questions

• RQ1 How can we derive tight, general amplification bounds for DP under arbitrary subsampling schemes?
• RQ2 Can α-divergence characterization, together with couplings, yield unified and optimal privacy amplification results?
• RQ3 How do advanced joint convexity and privacy profiles facilitate handling mixture outputs from subsampling?
• RQ4 What are the tight bounds for common subsampling methods (Poisson, WOR, WR) under different neighbor relations?
• RQ5 Are the obtained bounds provably tight via lower bounds across mechanisms?

Key findings

• Established a unified method to derive privacy amplification bounds that recovers all known results and yields new bounds.
• Introduced advanced joint convexity for α-divergences to bound mixed outputs of subsampling.
• Defined privacy profiles and group-privacy profiles to quantify DP guarantees across ε and δ.
• Derived tight amplification bounds for Poisson, WOR, and WR subsampling with remove/add-one (R) and substitute-one (S) neighborings.
• Provided a framework showing tightness via generic lower bounds and couplings.
• Table 1 summarizes the amplification bounds across common subsampling schemes and neighbor relations.