[Paper Review] Private Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication Overhead

Badih Ghazi, Ravi Kumar+2 more|arXiv (Cornell University)|Jun 8, 2021

Privacy-Preserving Technologies in Data19 citations

TL;DR

This paper presents communication-efficient differentially private algorithms for binary summation and histogram aggregation in the shuffled model, achieving near-optimal accuracy—arbitrarily close to central model performance—while maintaining per-user communication that matches non-private baselines. The approach uses a novel encoding strategy and hockey-stick divergence analysis to minimize error under strict privacy constraints.

ABSTRACT

Differential privacy (DP) is a formal notion for quantifying the privacy loss of algorithms. Algorithms in the central model of DP achieve high accuracy but make the strongest trust assumptions whereas those in the local DP model make the weakest trust assumptions but incur substantial accuracy loss. The shuffled DP model (Bittau et al., 2017; Erlingsson et al., 2019; Cheu et al., 2019) has recently emerged as a feasible middle ground between the central and local models, providing stronger trust assumptions than the former while promising higher accuracies than the latter. In this paper, we obtain practical communication-efficient algorithms in the shuffled DP model for two basic aggregation primitives used in machine learning: 1) binary summation, and 2) histograms over a moderate number of buckets. Our algorithms achieve accuracy that is arbitrarily close to that of central DP algorithms with an expected communication per user essentially matching what is needed without any privacy constraints! We demonstrate the practicality of our algorithms by experimentally comparing their performance to several widely-used protocols such as Randomized Response (Warner, 1965) and RAPPOR (Erlingsson et al., 2014).

Motivation & Objective

• Address the trade-off between communication efficiency, accuracy, and privacy in distributed data aggregation.
• Develop practical protocols for binary summation and histogram aggregation under the shuffled differential privacy model.
• Achieve accuracy close to the central model while requiring minimal trust, unlike the local model.
• Minimize per-user communication overhead to match non-private systems, enabling scalable deployment.
• Provide theoretical and empirical validation of privacy-accuracy-communication trade-offs in the shuffled model.

Proposed method

• Introduce a new encoding scheme that maps user inputs into messages with controlled noise, enabling accurate aggregation under shuffled DP.
• Leverage the hockey-stick divergence to formally bound the privacy loss (ε, δ) of the protocol, ensuring (ε, δ)-differential privacy.
• Use a grouped message analysis to reduce the complexity of computing privacy bounds, merging non-target messages into a single outcome.
• Apply pruning techniques to efficiently compute the expected positive part of sums of i.i.d. random variables in O(n·polylog(n/δ)) time.
• Design protocols for both binary summation (counting) and multi-bucket histograms using the same core framework.
• Validate the approach via experimental comparison with established protocols like Randomized Response and RAPPOR.

Experimental results

Research questions

• RQ1 Can we design a differentially private protocol for binary summation in the shuffled model that achieves near-central model accuracy with minimal communication?
• RQ2 How can we minimize the communication cost per user while maintaining strong privacy guarantees in the shuffled model?
• RQ3 What is the theoretical limit of accuracy for histogram aggregation under the shuffled model with bounded communication?
• RQ4 Can we achieve privacy-accuracy-communication trade-offs that outperform both local and central models?
• RQ5 How can we efficiently compute privacy parameters (ε, δ) for complex, grouped message distributions in the shuffled model?

Key findings

• The proposed protocol achieves estimation error arbitrarily close to that of central model algorithms, significantly outperforming local model baselines.
• Expected communication per user is asymptotically optimal—matching the communication cost of non-private protocols.
• For binary summation, the protocol achieves squared error within a constant factor of the discrete Laplace mechanism under central DP.
• The method enables near-optimal accuracy for histogram aggregation over a moderate number of buckets with minimal communication overhead.
• Empirical evaluation shows superior performance compared to Randomized Response and RAPPOR in terms of accuracy and communication efficiency.
• The use of grouped message analysis and efficient computation of expected positive parts enables practical deployment of the protocol at scale.