[Paper Review] Differentially private multivariate medians

Kelly Ramsay, Aukosh Jagannath+1 more|arXiv (Cornell University)|Oct 12, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper introduces a novel concentration inequality for the exponential mechanism applied to differentially private multivariate median estimation using depth functions. It establishes sharp finite-sample sample complexity bounds for depth-based medians—such as Tukey, spatial, and integrated dual depth—under minimal distributional assumptions, showing that privacy costs are minimal under heavy-tailed distributions like Cauchy.

ABSTRACT

Statistical tools which satisfy rigorous privacy guarantees are necessary for modern data analysis. It is well-known that robustness against contamination is linked to differential privacy. Despite this fact, using multivariate medians for differentially private and robust multivariate location estimation has not been systematically studied. We develop novel finite-sample performance guarantees for differentially private multivariate depth-based medians, which are essentially sharp. Our results cover commonly used depth functions, such as the halfspace (or Tukey) depth, spatial depth, and the integrated dual depth. We show that under Cauchy marginals, the cost of heavy-tailed location estimation outweighs the cost of privacy. We demonstrate our results numerically using a Gaussian contamination model in dimensions up to $d = 100$, and compare them to a state-of-the-art private mean estimation algorithm. As a by-product of our investigation, we prove concentration inequalities for the output of the exponential mechanism about the maximizer of the population objective function. This bound applies to objective functions that satisfy a mild regularity condition.

Motivation & Objective

• To address the lack of finite-sample performance guarantees for differentially private multivariate median estimation, particularly for robust depth-based medians.
• To close the gap in understanding the cost of privacy in multivariate location estimation beyond mean estimation, especially in heavy-tailed or contaminated models.
• To develop a general regularity condition—(K, F)-regularity—that enables sample complexity bounds for the exponential mechanism in multivariate settings.
• To demonstrate that privacy costs are negligible under heavy-tailed distributions like Cauchy, where robustness dominates over privacy overhead.
• To provide a practical framework for private depth-based median estimation with provable accuracy and computational feasibility in high dimensions (up to d=100).

Proposed method

• Proposes a new regularity condition, (K, F)-regularity, which controls both global sensitivity and sample complexity in the exponential mechanism.
• Derives a concentration inequality for the output of the exponential mechanism around the population maximizer of the objective function, under mild regularity and continuity assumptions.
• Applies the exponential mechanism to depth functions (e.g., Tukey, spatial, integrated dual depth), treating the depth maximizer as the multivariate median.
• Uses VC-theory and Laplace mechanism to bound the deviation of private depth values from their population counterparts, yielding high-probability error bounds.
• Employs a private sampling strategy for depth values at sample points, acknowledging that full vector release requires significant noise due to information leakage.
• Validates results numerically using a Gaussian contamination model in dimensions up to d=100, comparing against state-of-the-art private mean estimators.

Experimental results

Research questions

• RQ1 What is the finite-sample sample complexity of differentially private multivariate median estimation via depth functions?
• RQ2 How does the privacy cost compare to the robustness cost in heavy-tailed distributions such as Cauchy?
• RQ3 Can (K, F)-regularity be used to derive sharp sample complexity bounds for the exponential mechanism in multivariate settings?
• RQ4 What is the trade-off between privacy and accuracy when estimating depth values at sample points under differential privacy?
• RQ5 How do private depth-based medians perform relative to private mean estimators in high-dimensional, contaminated, or heavy-tailed data?

Key findings

• The proposed (K, F)-regularity condition enables sample complexity bounds of O(d log d) for depth-based medians, matching known lower bounds for private Gaussian mean estimation up to logarithmic factors.
• For Cauchy-distributed data, the cost of robustness (i.e., using a median) outweighs the cost of privacy, implying that privacy noise is negligible in such settings.
• Finite-sample error bounds for private depth values are derived using VC-theory and Laplace mechanism, yielding high-probability concentration: Pr(|D̃(x, μ̂n) - D(x, μ)| > t) ≤ (c1n/VC(F))^VC(F) e^{-c2nt²/K²} + e^{-nεt/2K}.
• The sample complexity for private depth-based medians is essentially sharp, as confirmed by comparison with lower bounds from Kamath et al. (2020) for private mean estimation.
• Private estimation of the full vector of depth values at sample points is infeasible due to high information leakage, requiring noise levels that exceed sampling error.
• A practical alternative is to sample N private median points using small ε, enabling downstream inference without full vector release.