[Paper Review] Iterative Constructions and Private Data Release

Anupam Gupta, Aaron Roth+1 more|arXiv (Cornell University)|Jul 19, 2011

Privacy-Preserving Technologies in Data12 references14 citations

TL;DR

This paper introduces a generic framework for differentially private query release using iterative database construction (IDC) algorithms, enabling improved accuracy in both interactive and non-interactive settings. It presents a new IDC based on Frieze-Kannan low-rank matrix decomposition that achieves better error bounds for releasing all cut queries on dense graphs, and a non-interactive synthetic data generation method with O(|V|^{1.5}) error, while identifying private rank-1 matrix approximation as a key open problem for further improvement.

ABSTRACT

In this paper we study the problem of approximately releasing the <em>cut function</em> of a graph while preserving differential privacy, and give new algorithms (and new analyses of existing algorithms) in both the interactive and non-interactive settings. Our algorithms in the interactive setting are achieved by revisiting the problem of releasing differentially private, approximate answers to a large number of queries on a database. We show that several algorithms for this problem fall into the same basic framework, and are based on the existence of objects which we call <em>iterative database construction</em> algorithms. We give a new generic framework in which new (efficient) IDC algorithms give rise to new (efficient) interactive private query release mechanisms. Our modular analysis simplifies and tightens the analysis of previous algorithms, leading to improved bounds. We then give a new IDC algorithm (and therefore a new private, interactive query release mechanism) based on the Frieze/Kannan low-rank matrix decomposition. This new release mechanism gives an improvement on prior work in a range of parameters where the size of the database is comparable to the size of the data universe (such as releasing all cut queries on dense graphs). We also give a non-interactive algorithm for efficiently releasing private <em>synthetic data</em> for graph cuts with error <em>O</em>(|<em>V</em>|1.5). Our algorithm is based on randomized response and a non-private implementation of the SDP-based, constant-factor approximation algorithm for cut-norm due to Alon and Naor. Finally, we give a reduction based on the IDC framework showing that an efficient, private algorithm for computing sufficiently accurate rank-1 matrix approximations would lead to an improved efficient algorithm for releasing private synthetic data for graph cuts. We leave finding such an algorithm as our main open problem.

Motivation & Objective

• To address the challenge of privately releasing all cut queries on a graph while preserving differential privacy.
• To develop a unified framework that generalizes existing private query release mechanisms and improves their accuracy bounds.
• To design an efficient, non-interactive algorithm for generating private synthetic graphs that approximately preserve the cut function.
• To identify private rank-1 matrix approximation as a key open problem that, if solved, would yield better private synthetic data for graph cuts.

Proposed method

• Introduces the concept of iterative database construction (IDC) algorithms as a unifying abstraction for private query release.
• Develops a new IDC algorithm based on the Frieze-Kannan low-rank matrix decomposition for efficient approximation of linear queries.
• Applies the IDC framework to derive improved accuracy bounds for both interactive and non-interactive query release mechanisms.
• Uses randomized response combined with a non-private SDP-based cut-norm approximation to construct a non-interactive private synthetic graph with O(|V|^{1.5}) error.
• Reduces the problem of private synthetic data generation for cuts to the existence of an efficient, private rank-1 matrix approximation algorithm.
• Employs a distinguisher-based mechanism that uses private oracle access and Laplace noise to ensure differential privacy.

Experimental results

Research questions

• RQ1 Can a generic framework unify and improve the analysis of existing private query release mechanisms for linear queries?
• RQ2 How can low-rank matrix decomposition be leveraged to design more accurate, differentially private query release algorithms for graph cuts?
• RQ3 What is the best possible error bound for non-interactive private synthetic data generation for graph cuts?
• RQ4 Can private rank-1 matrix approximation lead to improved private synthetic data for graph cuts?
• RQ5 Is it possible to design an efficient, private distinguisher for cut queries that enables better accuracy in synthetic data generation?

Key findings

• The proposed Frieze-Kannan-based IDC algorithm achieves an error bound of O(|E|^{1/4}|V| / ǫ^{1/2}) for releasing all cut queries on dense graphs, improving upon prior methods.
• The non-interactive mechanism based on randomized response and Alon-Naor's SDP-based cut-norm approximation achieves O(|V|^{1.5}) error for private synthetic graph generation.
• The generic IDC framework simplifies and tightens the analysis of prior mechanisms like median and multiplicative weights, leading to improved accuracy bounds.
• An (α, β)-accurate private query release mechanism is achieved for graph cuts using the multiplicative weights IDC with α = O(ǫ^{-1/2} (Tm log|V| log(1/δ))^{1/4}), where T is the distinguisher's error tolerance.
• An efficient private algorithm for rank-1 matrix approximation would yield improved private synthetic data for graph cuts, suggesting this as a key open problem.
• The framework shows that solving the private rank-1 approximation problem for symmetric matrices would directly yield better private synthetic data for graph cuts.