[Paper Review] Bounding, Concentrating, and Truncating: Unifying Privacy Loss Composition for Data Analytics

Mark Cesar, Ryan Rogers|arXiv (Cornell University)|Apr 15, 2020

Privacy-Preserving Technologies in Data15 references4 citations

TL;DR

This paper unifies privacy loss composition bounds for differential privacy by integrating pure DP, bounded range (BR), and concentrated DP (CDP) mechanisms under adaptive and non-adaptive selection. It provides optimal composition bounds that depend on mechanism ordering and introduces truncated Gaussian noise for private top-$k$ queries, significantly improving utility while maintaining tight privacy guarantees across mixed workloads.

ABSTRACT

Differential privacy (DP) provides rigorous privacy guarantees on individual's data while also allowing for accurate statistics to be conducted on the overall, sensitive dataset. To design a private system, first private algorithms must be designed that can quantify the privacy loss of each outcome that is released. However, private algorithms that inject noise into the computation are not sufficient to ensure individuals' data is protected due to many noisy results ultimately concentrating to the true, non-privatized result. Hence there have been several works providing precise formulas for how the privacy loss accumulates over multiple interactions with private algorithms. However, these formulas either provide very general bounds on the privacy loss, at the cost of being overly pessimistic for certain types of private algorithms, or they can be too narrow in scope to apply to general privacy systems. In this work, we unify existing privacy loss composition bounds for special classes of differentially private (DP) algorithms along with general DP composition bounds. In particular, we provide strong privacy loss bounds when an analyst may select pure DP, bounded range (e.g. exponential mechanisms), or concentrated DP mechanisms in any order. We also provide optimal privacy loss bounds that apply when an analyst can select pure DP and bounded range mechanisms in a batch, i.e. non-adaptively. Further, when an analyst selects mechanisms within each class adaptively, we show a difference in privacy loss between different, predetermined orderings of pure DP and bounded range mechanisms. Lastly, we compare the composition bounds of Laplace and Gaussian mechanisms based on histogram datasets.

Motivation & Objective

• To unify existing privacy loss composition bounds across pure DP, bounded range (BR), and concentrated DP (CDP) mechanisms in both adaptive and non-adaptive settings.
• To provide optimal composition bounds when combining pure DP and BR mechanisms in non-adaptive, homogeneous settings, interpolating between known bounds from prior works.
• To analyze how the ordering of pure DP and BR mechanisms affects privacy loss in adaptive compositions, revealing non-trivial dependencies.
• To compare Laplace and Gaussian mechanisms for private histograms, showing that truncated Gaussian noise can yield better utility with equivalent privacy.
• To develop new private top-$k$ algorithms using truncated Gaussian noise that improve on existing systems like LinkedIn’s.

Proposed method

• Introduces a general framework for adaptive composition where privacy parameters are pre-registered and selected without replacement, enabling flexible mechanism ordering.
• Derives optimal non-adaptive composition bounds for mixtures of pure DP and BR mechanisms with identical privacy parameters, enabling smooth interpolation between classical DP and BR bounds.
• Uses Rényi divergence and zCDP (zero-concentrated DP) composition to analyze privacy loss for Gaussian mechanisms, particularly under truncation.
• Proposes a new mechanism, TruncGauss, which applies truncated Gaussian noise to histogram counts, reducing tail risks and improving privacy-utility trade-offs.
• Applies post-processing invariance to threshold results after noise addition, ensuring no privacy loss increase.
• Employs union bounds over events $E$ and $E'$ to derive overall zCDP guarantees for multiple count updates in neighboring histograms.

Experimental results

Research questions

• RQ1 How does the ordering of pure DP and bounded range (BR) mechanisms affect the overall privacy loss in adaptive composition?
• RQ2 Can optimal composition bounds be derived for non-adaptive mixtures of pure DP and BR mechanisms with identical privacy parameters?
• RQ3 What is the privacy-utility trade-off when using truncated Gaussian noise instead of Laplace noise for private histogram queries?
• RQ4 How do the composition bounds of Laplace and Gaussian mechanisms compare under the same $ε$-$\delta$ privacy guarantee for private top-$k$ queries?
• RQ5 Can the optimal composition bound for heterogeneous mechanisms be computed efficiently, especially when mechanisms are adaptively selected?

Key findings

• The privacy loss in adaptive composition depends on the ordering of pure DP and BR mechanisms, with two BR mechanisms showing measurable differences based on placement.
• For non-adaptive settings with $m$ pure DP and $k-m$ BR mechanisms, the optimal composition bound smoothly interpolates between the $m=k$ (pure DP) and $m=0$ (BR-only) bounds.
• Truncated Gaussian noise reduces Rényi divergence between shifted Gaussians, leading to tighter privacy bounds than unbounded noise.
• The proposed TruncGauss mechanism achieves $\delta$-approximate $\tfrac{\Delta}{2\sigma^2}$-zCDP for $\Delta$ count changes in neighboring histograms, with improved utility.
• For private top-$k$ queries, the new Gaussian-based algorithms outperform Laplace-based methods in terms of noise magnitude and privacy efficiency under the same $(\varepsilon,\delta)$-DP constraint.
• The truncation level in TruncGauss is fixed and lower than in equivalent Laplace-based mechanisms, reducing tail risk and improving accuracy.