[Paper Review] PCKV: Locally Differentially Private Correlated Key-Value Data Collection with Optimized Utility

Xiaolan Gu, Ming Li+3 more|arXiv (Cornell University)|Nov 28, 2019

Privacy-Preserving Technologies in Data38 citations

TL;DR

This paper introduces PCKV, a locally differentially private framework for collecting correlated key-value data with optimized utility, including PCKV-UE and PCKV-GRR mechanisms and a Padding-and-Sampling sampling strategy.

ABSTRACT

Data collection under local differential privacy (LDP) has been mostly studied for homogeneous data. Real-world applications often involve a mixture of different data types such as key-value pairs, where the frequency of keys and mean of values under each key must be estimated simultaneously. For key-value data collection with LDP, it is challenging to achieve a good utility-privacy tradeoff since the data contains two dimensions and a user may possess multiple key-value pairs. There is also an inherent correlation between key and values which if not harnessed, will lead to poor utility. In this paper, we propose a locally differentially private key-value data collection framework that utilizes correlated perturbations to enhance utility. We instantiate our framework by two protocols PCKV-UE (based on Unary Encoding) and PCKV-GRR (based on Generalized Randomized Response), where we design an advanced Padding-and-Sampling mechanism and an improved mean estimator which is non-interactive. Due to our correlated key and value perturbation mechanisms, the composed privacy budget is shown to be less than that of independent perturbation of key and value, which enables us to further optimize the perturbation parameters via budget allocation. Experimental results on both synthetic and real-world datasets show that our proposed protocols achieve better utility for both frequency and mean estimations under the same LDP guarantees than state-of-the-art mechanisms.

Motivation & Objective

• Address the utility-privacy tradeoff in locally private collection of heterogeneous key-value data (keys with corresponding numeric values).
• Leverage correlation between keys and values to improve privacy budget composition and reduce noise.
• Provide non-interactive mechanisms with single-round mean estimation and improved sampling for large key domains.
• Enable near-optimal privacy budget allocation to minimize mean squared error under LDP guarantees.

Proposed method

• Propose a unified PCKV framework with two mechanisms: PCKV-UE (Unary Encoding based) and PCKV-GRR (Generalized Randomized Response based).
• Use a Padding-and-Sampling strategy to sample one key-value pair per user from possessed pairs, handling large key domains.
• Implement a correlated perturbation scheme where key and value perturbations are not independent, enabling tighter privacy budget composition.
• Derive a non-interactive mean estimator that computes mean via calibrated sum over calibrated frequency, with a corrected outlier approach at low epsilon.
• Provide theoretical analysis of expectation, MSE, and asymptotic unbiasedness for the estimators.
• Offer near-optimal budget allocation formulas to minimize MSE under the tight budget composition, with closed-form solutions.

Experimental results

Research questions

• RQ1 How to collect key-value data under LDP with high utility when users possess multiple key-value pairs?
• RQ2 Can correlated perturbations of key and value yield tighter privacy budget composition than independent perturbations?
• RQ3 Can a non-interactive, single-round mean estimator achieve asymptotic unbiasedness for key-value data under LDP?
• RQ4 What is the optimal budget allocation between key and value perturbations to minimize MSE for frequency and mean estimates?
• RQ5 Does Padding-and-Sampling improve utility for large key domains and real-world datasets?

Key findings

• The PCKV framework with PCKV-UE and PCKV-GRR achieves better utility than state-of-the-art mechanisms under the same LDP guarantees for both frequency and mean estimations.
• Correlated perturbation yields a tighter privacy budget composition than independent perturbation, enabling improved privacy-utility tradeoffs.
• A single-round mean estimator is proposed and shown to be asymptotically unbiased with a calibrated mean over calibrated frequency, plus an outlier correction at low epsilon.
• Padding-and-Sampling enhances utility for large key domains by sampling one key-value pair from possessed pairs.
• Near-optimal budget allocation in closed form outperforms naive equal-split allocation, improving MSE under given privacy constraints.
• Experimental results on synthetic and real-world data validate theoretical analyses and utility improvements over PrivKVM and related works.