[Paper Review] Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising

Borja Balle, Yu-Xiang Wang|arXiv (Cornell University)|May 16, 2018

Privacy-Preserving Technologies in DataComputer Science129 citations

TL;DR

The paper introduces an analytic Gaussian mechanism with CDF-based variance calibration for differential privacy and adds post-processing denoising to improve accuracy, especially in high dimensions.

ABSTRACT

The Gaussian mechanism is an essential building block used in multitude of differentially private data analysis algorithms. In this paper we revisit the Gaussian mechanism and show that the original analysis has several important limitations. Our analysis reveals that the variance formula for the original mechanism is far from tight in the high privacy regime ($\\varepsilon \ o 0$) and it cannot be extended to the low privacy regime ($\\varepsilon \ o \\infty$). We address these limitations by developing an optimal Gaussian mechanism whose variance is calibrated directly using the Gaussian cumulative density function instead of a tail bound approximation. We also propose to equip the Gaussian mechanism with a post-processing step based on adaptive estimation techniques by leveraging that the distribution of the perturbation is known. Our experiments show that analytical calibration removes at least a third of the variance of the noise compared to the classical Gaussian mechanism, and that denoising dramatically improves the accuracy of the Gaussian mechanism in the high-dimensional regime.

Motivation & Objective

• Identify limitations of the classical Gaussian mechanism in high and low privacy regimes.
• Develop an analytic Gaussian mechanism with exact CDF-based calibration for (ε,δ)-DP.
• Show that post-processing denoising can improve utility without compromising privacy.
• Provide practical algorithms and guidelines for implementing analytic calibration and denoising.
• Demonstrate utility gains through synthetic experiments and real-data applications.

Proposed method

• Develop an exact, CDF-based calibration condition for Gaussian noise ensuring (ε,δ)-DP (Theorem 8).
• Replace tail-bound based analysis with explicit Gaussian CDF expressions to compute optimal σ (Algorithm 1).
• Prove DP equivalence via a necessary-and-sufficient condition involving privacy loss random variables (Theorem 5).
• Present an implementable algorithm that computes σ using Φ and handles numerical stability (Theorem 9).
• Propose post-processing denoising strategies (Bayesian and minimax) leveraging known noise distribution (Theorem 10, Theorem 11, Theorem 12).
• Demonstrate denoising gains on mean estimation and NYC taxi heat maps (Section 5).

Experimental results

Research questions

• RQ1 Can the classical Gaussian mechanism be calibrated more tightly than ε-based tail bounds using the Gaussian CDF?
• RQ2 What is the exact σ required to achieve (ε,δ)-DP for a Gaussian mechanism when considering the full DP condition?
• RQ3 Does post-processing denoising of Gaussian-noised outputs improve utility without harming privacy, particularly in high dimensions?
• RQ4 How do analytic calibration and denoising perform on synthetic data and real datasets (e.g., NYC taxi heat maps)?

Key findings

• Analytic Gaussian Mechanism reduces noise variance by calibrating σ via Gaussian CDF, achieving tighter privacy-utility trade-offs (significant variance reductions as ε→0).
• Denoising the Gaussian-noised outputs with Bayes or minimax strategies yields substantial accuracy gains in high dimensions and for mean estimation.
• Soft-thresholding denoising adapts well to unknown function classes, offering near-optimal risk in Lp-balls and improving DP releases.
• Applied experiments show notable utility improvements for private mean estimation and NYC taxi heat maps with post-processing.