[Paper Review] Randomized Response Mechanisms for Differential Privacy Data Analysis: Bounds and Applications

Fei Ma, Ping Wang|arXiv (Cornell University)|Dec 14, 2021

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a principled framework for unbiased estimation using three-element randomized response (RR₃) mechanisms under local differential privacy (LDP), focusing on weighted bipartite graph analysis. It derives closed-form solutions for expectation and variance, optimizes parameters to minimize variance, and demonstrates that LDP mechanisms can inadvertently consume privacy budget, leading to stronger privacy than sequential composition.

ABSTRACT

Randomized response, as a basic building-block for differentially private mechanism, has given rise to great interest and found various potential applications in science communities. In this work, we are concerned with three-elements randomized response (RR$_{3}$) along with relevant applications to the analysis of weighted bipartite graph upon differentially private guarantee. We develop a principled framework for estimating statistics produced by RR$_{3}$-based mechanisms, and then prove the corresponding estimations to be unbiased. At the same time, we study in detail several fundamental and significant members in RR$_{3}$ family, and derive the closed-form solutions to unbiased estimations. Next, we show potential applications of several RR$_{3}$-based mechanisms into the estimation of average degree and average weighted value on weighted bipartite graph when requiring local differential privacy guarantee. In the meantime, we determine the lower bounds for choice of relevant parameters by minimizing variance of statistics in order to design optimal RR$_{3}$-based local differential private mechanisms, with which we optimize previous protocols in the literature and put forward a version that achieves the tight bound. Last but most importantly, we observe that in the analysis of relational data such as weighted bipartite graph, a portion of privacy budget in local differential private mechanism is sometimes "consumed" by mechanism itself accidentally, resulting to a more stronger privacy guarantee than we would get by simply sequential compositions.

Motivation & Objective

• To develop a unified framework for estimating statistics from three-element randomized response (RR₃) mechanisms under local differential privacy (LDP).
• To derive closed-form solutions for unbiased estimation of key statistics, such as average degree and average weighted value, in weighted bipartite graphs.
• To optimize RR₃-based mechanisms by minimizing variance, thereby achieving tighter privacy-utility trade-offs.
• To investigate the counterintuitive phenomenon where LDP mechanisms inadvertently consume privacy budget, resulting in stronger privacy than sequential composition would imply.
• To improve upon existing protocols by proposing an optimal RR₃ mechanism that achieves the theoretical lower bound on variance.

Proposed method

• Applies maximum likelihood estimation (MLE) to derive closed-form expressions for the expectation and variance of RR₃-based estimators.
• Analyzes four fundamental RR₃ mechanisms: EWRR₃, RR₃†, RR₃‡, and RR₃♣, to characterize their statistical properties.
• Derives optimal parameter settings for RR₃ mechanisms by minimizing the variance of the estimators, ensuring the tightest possible privacy-utility balance.
• Applies the optimized RR₃ mechanisms to estimate average degree and average weighted value in weighted bipartite graphs under LDP.
• Demonstrates that the privacy budget is not fully consumed by the query but partially consumed by the mechanism itself, leading to stronger privacy than sequential composition would suggest.
• Validates the theoretical findings through analysis of relational data structures, particularly weighted bipartite graphs, under LDP constraints.

Experimental results

Research questions

• RQ1 What are the closed-form solutions for unbiased estimation of average degree and average weighted value in weighted bipartite graphs using RR₃ mechanisms?
• RQ2 How can the parameters of RR₃ mechanisms be optimized to minimize variance and achieve the tightest privacy-utility trade-off?
• RQ3 What is the impact of mechanism-induced privacy budget consumption on the overall privacy guarantee in LDP settings?
• RQ4 How do RR₃-based mechanisms compare to existing protocols in terms of estimation accuracy and privacy efficiency?
• RQ5 Can the observed phenomenon of 'accidental' privacy budget consumption in LDP mechanisms lead to stronger privacy than sequential composition?

Key findings

• The paper derives exact closed-form solutions for the expectation and variance of estimators under four fundamental RR₃ mechanisms: EWRR₃, RR₃†, RR₃‡, and RR₃♣.
• Optimal parameter settings for EWRR₃ are derived by minimizing variance, resulting in a mechanism that achieves the theoretical lower bound on estimation error.
• The analysis reveals that a portion of the privacy budget is consumed by the mechanism itself during data collection, leading to stronger privacy guarantees than those predicted by sequential composition.
• The proposed optimized RR₃ mechanism outperforms previous protocols in the literature, achieving tighter bounds on estimation variance.
• The framework enables accurate estimation of average degree and average weighted value in weighted bipartite graphs while preserving local differential privacy.
• The findings suggest that mechanism design in LDP should account for internal privacy consumption, as it can lead to improved privacy guarantees beyond standard composition theorems.