[Paper Review] Perturbed Gibbs Samplers for Synthetic Data Release

Yubin Park, Joydeep Ghosh|arXiv (Cornell University)|Dec 18, 2013

Privacy-Preserving Technologies in Data21 references3 citations

TL;DR

This paper proposes the Perturbed Gibbs Sampler (PeGS), a differentially private synthetic data generator for high-dimensional categorical data that uses feature hashing and non-parametric modeling to preserve statistical utility while ensuring quantifiable privacy. PeGS achieves strong privacy guarantees via differential privacy and l-diversity, with empirical validation on California Patient Discharge data showing minimal distortion in marginal, conditional, and regression-based statistics.

ABSTRACT

We propose a categorical data synthesizer with a quantifiable disclosure risk. Our algorithm, named Perturbed Gibbs Sampler, can handle high-dimensional categorical data that are often intractable to represent as contingency tables. The algorithm extends a multiple imputation strategy for fully synthetic data by utilizing feature hashing and non-parametric distribution approximations. California Patient Discharge data are used to demonstrate statistical properties of the proposed synthesizing methodology. Marginal and conditional distributions, as well as the coefficients of regression models built on the synthesized data are compared to those obtained from the original data. Intruder scenarios are simulated to evaluate disclosure risks of the synthesized data from multiple angles. Limitations and extensions of the proposed algorithm are also discussed.

Motivation & Objective

• To develop a scalable synthetic data generator for high-dimensional categorical data that is infeasible to represent as contingency tables.
• To ensure quantifiable disclosure risk through formal privacy guarantees, including ε-differential privacy and l-diversity.
• To maintain high statistical utility of synthetic data by preserving marginal and conditional distributions and regression coefficients.
• To evaluate privacy risks through simulated intruder attacks, including population uniqueness and probabilistic disclosure risks.
• To explore the trade-off between privacy and utility in synthetic data generation, especially under strict privacy budgets.

Proposed method

• The method employs a block Gibbs sampling framework with non-parametric conditional probability estimation using feature hashing to scale to high-dimensional data.
• Each variable is imputed conditionally using a non-parametric kernel density approximation based on rows sharing the same hash key of the remaining variables.
• A perturbation step is applied to the conditional probabilities using Laplace noise to achieve ε-differential privacy.
• The algorithm uses a hashing scheme to group similar data rows, enabling efficient computation and reducing the need for full contingency tables.
• The method supports multiple imputations, allowing for robust statistical inference on synthetic datasets.
• Privacy is formally analyzed using ε-differential privacy and l-diversity, with additional risk assessment via population uniqueness and indirect matching probabilities.

Experimental results

Research questions

• RQ1 Can a synthetic data generator maintain statistical fidelity to the original data while ensuring formal privacy guarantees in high-dimensional categorical data?
• RQ2 How does the use of feature hashing and non-parametric modeling affect the utility and scalability of synthetic data generation?
• RQ3 To what extent does the perturbation step in PeGS reduce disclosure risks compared to traditional methods?
• RQ4 How do different privacy parameters (ε) affect the balance between utility and privacy in the synthetic data?
• RQ5 Can the proposed method outperform existing techniques in terms of both privacy and statistical similarity to the original data?

Key findings

• The Perturbed Gibbs Sampler successfully generates synthetic data with minimal distortion in marginal and conditional distributions compared to the original California Patient Discharge dataset.
• Regression coefficients estimated on synthetic data closely match those from the original data, demonstrating strong statistical utility.
• The algorithm increases population uniqueness, which is a key differentiator from k-anonymity or l-diversity methods that reduce uniqueness.
• Lower ε values led to higher absolute errors in distributional comparisons, supporting the use of ε as a meaningful privacy-utility trade-off measure.
• The method achieved strong privacy guarantees under ε-differential privacy and l-diversity, with simulated attacks showing low success rates in record re-identification.
• The hashing mechanism contributes to privacy beyond computational efficiency, though a tighter formal privacy analysis of hashing is needed in future work.