[Paper Review] Exact Optimality of Communication-Privacy-Utility Tradeoffs in Distributed Mean Estimation

Berivan Isik, Weining Chen+3 more|arXiv (Cornell University)|Jun 8, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a communication-privacy-utility optimal mechanism for distributed mean estimation under shared randomness, introducing Random Rotating Simplex Coding (RRSC) with k-closest encoding. It proves exact optimality by showing that rotationally symmetric codebooks—specifically, randomly rotated simplices—are necessary and sufficient, achieving optimal mean squared error with minimal communication and strict local differential privacy constraints.

ABSTRACT

We study the mean estimation problem under communication and local differential privacy constraints. While previous work has proposed \emph{order}-optimal algorithms for the same problem (i.e., asymptotically optimal as we spend more bits), \emph{exact} optimality (in the non-asymptotic setting) still has not been achieved. In this work, we take a step towards characterizing the \emph{exact}-optimal approach in the presence of shared randomness (a random variable shared between the server and the user) and identify several conditions for \emph{exact} optimality. We prove that one of the conditions is to utilize a rotationally symmetric shared random codebook. Based on this, we propose a randomization mechanism where the codebook is a randomly rotated simplex -- satisfying the properties of the \emph{exact}-optimal codebook. The proposed mechanism is based on a $k$-closest encoding which we prove to be \emph{exact}-optimal for the randomly rotated simplex codebook.

Motivation & Objective

• Address the gap in exact optimality for distributed mean estimation under joint communication and local differential privacy (LDP) constraints.
• Characterize the necessary conditions for exact optimality when shared randomness is available.
• Develop a mechanism that achieves exact optimality in non-asymptotic settings, not just order optimality.
• Design a codebook generation strategy that maximizes codeword separation while satisfying LDP and communication constraints.
• Demonstrate that the proposed scheme asymptotically approaches PrivUnit, the known optimal mechanism, under increasing communication budget.

Proposed method

• Formulate the problem as a lossy compression task under ε-LDP constraints using shared randomness.
• Introduce 'random coding with a codebook' as the optimal framework, where server and user jointly generate a codebook of size M=2^b using shared randomness.
• Prove that the codebook-generating distribution must be rotationally symmetric for exact optimality.
• Propose Random Rotating Simplex Coding (RRSC), where the codebook is a uniformly random rotation of a regular simplex.
• Implement k-closest encoding: the user selects the k codewords closest to its input vector with higher sampling probability.
• Show that this encoding achieves exact optimality for the RRSC codebook under ε-LDP and b-bit communication constraints.

Experimental results

Research questions

• RQ1 What conditions on the codebook-generating distribution are necessary and sufficient for exact optimality in distributed mean estimation with shared randomness?
• RQ2 Can a mechanism be constructed that achieves exact optimality (not just order optimality) under joint communication and ε-LDP constraints?
• RQ3 How does the use of rotationally symmetric codebooks—specifically, randomly rotated simplices—improve utility and privacy tradeoffs?
• RQ4 What is the relationship between the proposed RRSC mechanism and existing order-optimal mechanisms like PrivUnit or MMRC?
• RQ5 Can the proposed scheme be extended to cases where the number of codewords M exceeds the input dimension d?

Key findings

• The exact optimal codebook-generating distribution under shared randomness must be rotationally symmetric, i.e., invariant under random rotations.
• Random Rotating Simplex Coding (RRSC) with k-closest encoding achieves exact optimality for the given communication and privacy constraints.
• The proposed mechanism achieves mean squared error (MSE) that matches the theoretical lower bound, making it exactly optimal in the non-asymptotic regime.
• When the communication budget satisfies b ≤ log d (i.e., M < d), the RRSC codebook is valid and achieves optimal performance due to maximal codeword separation.
• As the communication budget b increases, the proposed scheme asymptotically converges to PrivUnit, the known optimal mechanism under ε-LDP.
• For M ≥ d, the scheme can be extended using nearly maximally separated M vectors on the sphere, maintaining optimality under the same constraints.