[Paper Review] Pure Differentially Private Summation from Anonymous Messages

Badih Ghazi, Noah Golowich+4 more|arXiv (Cornell University)|Jan 1, 2020

Privacy-Preserving Technologies in DataComputer Science35 references14 citations

TL;DR

This paper presents the first pure differentially private protocols in the shuffled model for binary and real-number summation with constant error. It achieves this by designing a novel multi-message shuffling protocol where each user sends Oϵ(log n) bits for binary summation and Oϵ(log³n) bits for real summation, while proving a tight Ωϵ(√log n) communication lower bound, establishing separations between pure and approximate DP, and between the shuffled and central models.

ABSTRACT

The shuffled (aka anonymous) model has recently generated significant interest as a candidate distributed privacy framework with trust assumptions better than the central model but with achievable errors smaller than the local model. We study pure differentially private (DP) protocols in the shuffled model for summation, a basic and widely used primitive: - For binary summation where each of n users holds a bit as an input, we give a pure $ε$-DP protocol for estimating the number of ones held by the users up to an error of $O_ε(1)$, and each user sends $O_ε(\log n)$ messages each of 1 bit. This is the first pure protocol in the shuffled model with error $o(\sqrt{n})$ for constant $ε$. Using this protocol, we give a pure $ε$-DP protocol that performs summation of real numbers in $[0, 1]$ up to an error of $O_ε(1)$, and where each user sends $O_ε(\log^3 n)$ messages each of $O(\log\log n)$ bits. - In contrast, we show that for any pure $ε$-DP protocol for binary summation in the shuffled model having absolute error $n^{0.5-Ω(1)}$, the per user communication has to be at least $Ω_ε(\sqrt{\log n})$ bits. This implies the first separation between the (bounded-communication) multi-message shuffled model and the central model, and the first separation between pure and approximate DP protocols in the shuffled model. To prove our lower bound, we consider (a generalization of) the following question: given $γ$ in $(0, 1)$, what is the smallest m for which there are two random variables $X^0, X^1$ supported on $\{0, \dots ,m\}$ such that (i) the total variation distance between $X^0$ and $X^1$ is at least $1-γ$, and (ii) the moment generating functions of $X^0$ and $X^1$ are within a constant factor of each other everywhere? We show that the answer is $m = Θ(\sqrt{\log(1/γ)})$.

Motivation & Objective

• Develop the first pure differentially private protocol for binary summation in the shuffled model with error o(√n) and constant absolute error.
• Extend the binary summation protocol to handle real numbers in [0,1] with constant error and efficient communication.
• Establish a communication lower bound for pure DP in the shuffled model to separate it from both the central model and approximate DP protocols.
• Address the fundamental question of the minimal communication required for pure DP in the shuffled model, particularly for summation.
• Provide a new analytical framework for bounding the total variation distance and moment generating functions of discrete distributions, which may have broader applications.

Proposed method

• Design a multi-message shuffling protocol where each user encodes their bit input into Oϵ(log n) anonymous messages, each a single bit, using a randomized response with carefully tuned noise.
• Use the shuffler to permute all messages, ensuring that the analyzer cannot link messages to users, thus achieving pure differential privacy.
• Apply composition theorems to bound the total privacy loss across multiple bit positions in the real-number summation protocol, allocating privacy budget to minimize error.
• Prove a lower bound on communication by analyzing the moment generating function (MGF) ratio of two discrete distributions and relating it to total variation distance.
• Generalize the analysis to bound the minimal m such that two distributions on {0,...,m} have total variation distance ≥1−γ and MGF ratio within a constant factor, showing m = Θ(√log(1/γ)).
• Construct a real-number summation protocol by independently applying the binary protocol to each bit of the base-2 representation of inputs, with privacy budget allocation to minimize error.

Experimental results

Research questions

• RQ1 Can pure differentially private protocols in the shuffled model achieve constant error for binary summation with sublinear communication?
• RQ2 What is the optimal communication complexity for pure DP in the shuffled model for binary and real-number summation?
• RQ3 Is there a provable separation between pure and approximate differentially private protocols in the shuffled model?
• RQ4 Can the communication lower bound for pure DP in the shuffled model be established using moment generating function analysis?
• RQ5 Does the multi-message shuffled model offer a strict advantage over the central model in terms of communication efficiency for pure DP?

Key findings

• The paper presents a pure ϵ-differentially private protocol for binary summation in the shuffled model with absolute error Oϵ(1), where each user sends Oϵ(log n) messages of one bit each.
• For real-number summation in [0,1], the protocol achieves an expected error of O(√log(1/ϵ)/ϵ³/²) with each user sending Oϵ(log³n) messages of O(log log n) bits.
• A communication lower bound of Ωϵ(√log n) bits per user is proven for any pure ϵ-DP protocol for binary summation with error n⁰.⁵⁻ᴼ⁽¹⁾, establishing a separation from the central model.
• The lower bound is derived by analyzing the minimal m such that two distributions on {0,...,m} have total variation distance ≥1−γ and MGF ratio within a constant factor, showing m = Θ(√log(1/γ)).
• The protocol demonstrates the first separation between pure and approximate DP in the shuffled model, as approximate DP protocols can achieve sub-constant error with lower communication.
• The real-number summation protocol uses independent binary protocols per bit position, with privacy budget allocation to minimize error, achieving a communication complexity of Oϵ(log³n) per user.