[论文解读] Pan-private Algorithms: When Memory Does Not Help

Darakhshan Mir, S. Muthukrishnan+2 more|arXiv (Cornell University)|Sep 8, 2010

Privacy-Preserving Technologies in Data参考文献 12被引用 3

一句话总结

本文提出改进的泛私有流算法，用于基础计数任务——不同计数和高频项检测，利用噪声化草图技术确保即使内部内存被泄露，也能实现差分隐私。研究证明，无界内存不会显著提升准确性，通过噪声解码建立紧致下界，并在不同计数估计中实现最优加法误差界，高频项检测实现最坏情况 O(k) 近似。

ABSTRACT

Consider updates arriving online in which the $t$th input is $(i_t,d_t)$, where $i_t$'s are thought of as IDs of users. Informally, a randomized function $f$ is {\em differentially private} with respect to the IDs if the probability distribution induced by $f$ is not much different from that induced by it on an input in which occurrences of an ID $j$ are replaced with some other ID $k$ Recently, this notion was extended to {\em pan-privacy} where the computation of $f$ retains differential privacy, even if the internal memory of the algorithm is exposed to the adversary (say by a malicious break-in or by fiat by the government). This is a strong notion of privacy, and surprisingly, for basic counting tasks such as distinct counts, heavy hitters and others, Dwork et al~\cite{dwork-pan} present pan-private algorithms with reasonable accuracy. The pan-private algorithms are nontrivial, and rely on sampling. We reexamine these basic counting tasks and show improved bounds. In particular, we estimate the distinct count $\Dt$ to within $(1\pm \eps)\Dt \pm O(\polylog m)$, where $m$ is the number of elements in the universe. This uses suitably noisy statistics on sketches known in the streaming literature. We also present the first known lower bounds for pan-privacy with respect to a single intrusion. Our lower bounds show that, even if allowed to work with unbounded memory, pan-private algorithms for distinct counts can not be significantly more accurate than our algorithms. Our lower bound uses noisy decoding. For heavy hitter counts, we present a pan private streaming algorithm that is accurate to within $O(k)$ in worst case; previously known bound for this problem is arbitrarily worse. An interesting aspect of our pan-private algorithms is that, they deliberately use very small (polylogarithmic) space and tend to be streaming algorithms, even though using more space is not forbidden.

研究动机与目标

• 解决一个基本问题：在泛私有算法中，对于基础计数任务，无界内存是否能显著提升准确性？
• 设计在内部状态被敌手暴露时仍能维持差分隐私的泛私有流算法。
• 为泛私有算法在不同计数估计中的准确性建立紧致下界，证明内存无法提升性能。
• 开发首个在最坏情况 O(k) 近似下实现高频项计数的泛私有流算法。
• 探索在流式环境中泛私有的极限，特别是涉及正负更新的各类问题。

提出的方法

• 利用流计算文献中的噪声化草图技术，实现 (1±ε)D ± O(polylog m) 加法误差的差异计数估计。
• 在草图上应用随机响应机制，确保即使内部状态被泄露，也能实现泛私有性。
• 采用噪声解码方法证明下界，表明任何泛私有算法都无法在无界内存下实现显著优于所提算法的准确性。
• 通过两两独立哈希与噪声注入，实现内积和截断频率矩（T₂）的随机估计器。
• 依赖霍夫丁不等式，限制噪声草图过程中的估计误差。
• 放宽对已知 F₁ 的假设：通过运行多个并行实例，使用指数增长的 F₁ 估计值，在查询时选择最匹配的实例。

实验结果

研究问题

• RQ1 若赋予无界内存，泛私有算法能否在不同计数估计中实现显著更优的准确性？
• RQ2 在存储无限的条件下，泛私有算法在估计不同计数时的准确性是否存在根本性限制？
• RQ3 能否设计出具有有界最坏情况误差的泛私有流算法来估计高频项计数？
• RQ4 使用噪声草图如何影响泛私有计算中隐私与效用之间的权衡？
• RQ5 标准流式技术在内存暴露条件下，能在多大程度上被适配以满足泛私有性？

主要发现

• 本文通过噪声解码论证证明，即使在无界内存下，泛私有算法在不同计数估计中也无法实现优于 O(polylog m) 的加法误差。
• 提出一种泛私有流算法，可在 (1±ε)D(t) ± O(polylog m) 范围内估计不同计数 D(t)，达到信息论下界。
• 对于高频项计数，本文设计的泛私有算法具有最坏情况 O(k) 近似误差，优于此前已知的任意更差界。
• 所提算法本质上为流式算法，仅使用多项式对数空间，尽管模型并未禁止使用更大内存。
• 本文证明，使用噪声草图与随机响应机制可实现泛私有性，且无需依赖小内存；内存大小无法提升准确性。
• 结果表明，截断内积估计器 ˜(a·a′)(τ) 既满足泛私有性又无偏，其误差受霍夫丁不等式约束。