[论文解读] Circulant Matrices and Differential Privacy.

Jalaj Upadhyay|arXiv (Cornell University)|Jan 1, 2014

Privacy-Preserving Technologies in Data参考文献 50被引用 3

一句话总结

该论文通过证明一类广义的循环随机投影矩阵——仅需 n 个高斯样本和 n 个伯努利试验——在保持 Johnson-Lindenstrauss 有效性的同时，能够保持差分隐私。该方法实现了 O(n log n) 的矩阵-向量乘法时间复杂度，相较于以往工作，在切分查询上提升了 O(n^o(1)) 的运行时间因子，在协方差查询上提升了 O(n^0.38) 的因子，且无需依赖图稀疏化技术。

ABSTRACT

This paper resolves an open problem raised by Blocki et al. (FOCS 2012), i.e., whether other variants of the Johnson-Lindenstrauss transform preserves differential privacy or not? We prove that a general class of random projection matrices that satisfies the Johnson-Lindenstrauss lemma also preserves dif-ferential privacy. This class of random projection matrices requires only n Gaussian samples and n Bernoulli trials and allows matrix-vector multiplication in O(n log n) time. In this respect, this work un-conditionally improves the run time of Blocki et al. (FOCS 2012) without using the graph sparsification trick of Upadhyay (ASIACRYPT 2013). For the metric of measuring randomness, we stick to the norm used by earlier researchers who studied variants of the Johnson-Lindenstrauss transform and its applica-tions, i.e., count the number of random samples made. In concise, we improve the sampling complexity by quadratic factor, and the run time of cut queries by an O(no(1)) factor and that of covariance queries by an O(n0.38) factor. Our proof for both the privacy and utility guarantee uses several new ideas. In order to improve the dimension bound, we use some known results from the domain of statistical model selection. This makes our proof short and elegant, relying just on one basic concentration inequality. For the privacy proof, even though our mechanism closely resembles that of Blocki et al. (FOCS 2012) and Upadhyay (ASIACRYPT 2013), we cannot use their proof idea. This is because the projection matrices we are inter-ested in introduces non-trivial correlations between any two rows of the published matrix, and, therefore, we cannot invoke the composition theorem of Dwork, Rothblum and Vadhan (STOC 2009). We argue that the published matrix is not r-multivariate distribution; rather one matrix-variate distribution. We compute the distribution of the published matrix and then prove it preserves differential privacy.

研究动机与目标

• 解决 Blocki 等人（FOCS 2012）提出的开放问题：即 Johnson-Lindenstrauss 变体是否保持差分隐私。
• 设计一类随机投影矩阵，使其在满足 Johnson-Lindenstrauss 引理的同时，同时保持差分隐私与有效性。
• 降低采样复杂度，并改进隐私保护查询（如切分查询与协方差查询）的运行时间。
• 提出一种新颖的隐私证明，避免因投影矩阵中行之间存在非平凡相关性而依赖组合定理。
• 通过集中不等式与矩阵变量子分布分析，统一隐私与有效性保证。

提出的方法

• 利用 n 个高斯样本和 n 个伯努利试验构造一类循环随机投影矩阵。
• 利用统计模型选择中的已知结果，收紧 Johnson-Lindenstrauss 引理中的维数界。
• 通过单一基本集中不等式证明有效性，相比以往工作简化了分析。
• 由于行之间的相关性，将发布矩阵建模为矩阵变量子分布，而非多元分布。
• 通过精确计算发布矩阵的分布，建立差分隐私，而无需依赖组合定理。
• 采用新颖的隐私证明策略，考虑行之间的依赖关系，避免以往机制中使用的假设。

实验结果

研究问题

• RQ1 循环随机投影矩阵是否能在满足 Johnson-Lindenstrauss 引理的同时保持差分隐私？
• RQ2 为何标准组合定理不适用于此类投影矩阵？隐私如何在其他方式下证明？
• RQ3 能否在不使用图稀疏化的情况下，改进差分隐私降维的采样复杂度与运行时间？
• RQ4 与 i.i.d. 行模型相比，发布矩阵的矩阵变量子结构如何影响隐私分析？
• RQ5 该方法在切分查询与协方差查询中，运行时间的定量提升是多少？

主要发现

• 与以往工作相比，所提方法将采样复杂度降低了平方因子。
• 矩阵-向量乘法的运行时间为 O(n log n)，优于以往方法。
• 切分查询的运行时间提升了 O(n^o(1)) 因子，协方差查询的运行时间提升了 O(n^0.38) 因子。
• 尽管存在非平凡的行相关性，隐私证明依然有效，而标准组合定理因此失效。
• 分析仅依赖单一集中不等式，使证明简洁而优雅。
• 该机制通过将发布矩阵建模为矩阵变量子分布，实现精确的分布计算，从而实现差分隐私。