[Paper Review] Circulant Matrices and Differential Privacy.

Jalaj Upadhyay|arXiv (Cornell University)|Jan 1, 2014

Privacy-Preserving Technologies in Data50 references3 citations

TL;DR

This paper resolves an open problem by proving that a broad class of circulant random projection matrices—requiring only n Gaussian samples and n Bernoulli trials—preserves differential privacy while maintaining Johnson-Lindenstrauss utility. It achieves O(n log n) matrix-vector multiplication time, improving runtime over prior work by an O(n^o(1)) factor for cut queries and O(n^0.38) for covariance queries, without relying on graph sparsification.

ABSTRACT

This paper resolves an open problem raised by Blocki et al. (FOCS 2012), i.e., whether other variants of the Johnson-Lindenstrauss transform preserves differential privacy or not? We prove that a general class of random projection matrices that satisfies the Johnson-Lindenstrauss lemma also preserves dif-ferential privacy. This class of random projection matrices requires only n Gaussian samples and n Bernoulli trials and allows matrix-vector multiplication in O(n log n) time. In this respect, this work un-conditionally improves the run time of Blocki et al. (FOCS 2012) without using the graph sparsification trick of Upadhyay (ASIACRYPT 2013). For the metric of measuring randomness, we stick to the norm used by earlier researchers who studied variants of the Johnson-Lindenstrauss transform and its applica-tions, i.e., count the number of random samples made. In concise, we improve the sampling complexity by quadratic factor, and the run time of cut queries by an O(no(1)) factor and that of covariance queries by an O(n0.38) factor. Our proof for both the privacy and utility guarantee uses several new ideas. In order to improve the dimension bound, we use some known results from the domain of statistical model selection. This makes our proof short and elegant, relying just on one basic concentration inequality. For the privacy proof, even though our mechanism closely resembles that of Blocki et al. (FOCS 2012) and Upadhyay (ASIACRYPT 2013), we cannot use their proof idea. This is because the projection matrices we are inter-ested in introduces non-trivial correlations between any two rows of the published matrix, and, therefore, we cannot invoke the composition theorem of Dwork, Rothblum and Vadhan (STOC 2009). We argue that the published matrix is not r-multivariate distribution; rather one matrix-variate distribution. We compute the distribution of the published matrix and then prove it preserves differential privacy.

Motivation & Objective

• To resolve an open problem posed by Blocki et al. (FOCS 2012) on whether Johnson-Lindenstrauss variants preserve differential privacy.
• To design a class of random projection matrices that maintains both differential privacy and utility under the Johnson-Lindenstrauss lemma.
• To reduce sampling complexity and improve runtime for privacy-preserving queries such as cut and covariance queries.
• To provide a novel privacy proof that avoids composition theorems due to non-trivial row correlations in the projection matrix.
• To unify privacy and utility guarantees using concentration inequalities and matrix-variate distribution analysis.

Proposed method

• Construct a class of circulant random projection matrices using n Gaussian samples and n Bernoulli trials.
• Leverage known results from statistical model selection to tighten the dimensionality bound in the Johnson-Lindenstrauss lemma.
• Prove utility using a single basic concentration inequality, simplifying the analysis compared to prior work.
• Model the published matrix as a matrix-variate distribution rather than a multivariate one, due to row correlations.
• Compute the exact distribution of the published matrix to establish differential privacy without relying on composition theorems.
• Use a novel privacy proof strategy that accounts for dependencies between rows, avoiding assumptions used in prior mechanisms.

Experimental results

Research questions

• RQ1 Can circulant random projection matrices preserve differential privacy while satisfying the Johnson-Lindenstrauss lemma?
• RQ2 Why do standard composition theorems fail for this class of projection matrices, and how can privacy be proven otherwise?
• RQ3 Can the sampling complexity and runtime of differentially private dimensionality reduction be improved without using graph sparsification?
• RQ4 How does the matrix-variate nature of the published matrix affect privacy analysis compared to i.i.d. row models?
• RQ5 What is the quantitative improvement in runtime for cut and covariance queries using this approach?

Key findings

• The proposed method reduces sampling complexity by a quadratic factor compared to prior work.
• Matrix-vector multiplication runs in O(n log n) time, improving over previous methods.
• The runtime for cut queries improves by an O(n^o(1)) factor, and for covariance queries by an O(n^0.38) factor.
• The privacy proof is valid despite non-trivial row correlations, which invalidate standard composition theorems.
• The analysis relies on a single concentration inequality, making the proof both short and elegant.
• The mechanism achieves differential privacy by modeling the published matrix as a matrix-variate distribution, enabling precise distributional computation.