[论文解读] Privately Learning Thresholds: Closing the Exponential Gap

Haim Kaplan, Katrina Ligett+3 more|arXiv (Cornell University)|Nov 22, 2019

Privacy-Preserving Technologies in Data参考文献 11被引用 4

一句话总结

该论文通过提出一种基于递归哈希和路径构造的新颖算法，弥合了差分隐私学习阈值函数的最优已知下界与上界之间的指数级差距。该算法实现了Õ((log* |X|)^1.5)的样本复杂度，显著优于以往工作，几乎确定了该问题在近似差分隐私下的渐近样本复杂度。

ABSTRACT

We study the sample complexity of learning threshold functions under the constraint of differential privacy. It is assumed that each labeled example in the training data is the information of one individual and we would like to come up with a generalizing hypothesis $h$ while guaranteeing differential privacy for the individuals. Intuitively, this means that any single labeled example in the training data should not have a significant effect on the choice of the hypothesis. This problem has received much attention recently; unlike the non-private case, where the sample complexity is independent of the domain size and just depends on the desired accuracy and confidence, for private learning the sample complexity must depend on the domain size $X$ (even for approximate differential privacy). Alon et al. (STOC 2019) showed a lower bound of $Ω(\log^*|X|)$ on the sample complexity and Bun et al. (FOCS 2015) presented an approximate-private learner with sample complexity $ ilde{O}\left(2^{\log^*|X|} ight)$. In this work we reduce this gap significantly, almost settling the sample complexity. We first present a new upper bound (algorithm) of $ ilde{O}\left(\left(\log^*|X| ight)^2 ight)$ on the sample complexity and then present an improved version with sample complexity $ ilde{O}\left(\left(\log^*|X| ight)^{1.5} ight)$. Our algorithm is constructed for the related interior point problem, where the goal is to find a point between the largest and smallest input elements. It is based on selecting an input-dependent hash function and using it to embed the database into a domain whose size is reduced logarithmically; this results in a new database, an interior point of which can be used to generate an interior point in the original database in a differentially private manner.

研究动机与目标

• 弥合差分隐私学习阈值函数的样本复杂度的最优已知下界与上界之间的指数级差距。
• 设计一种私有算法，实现在近似差分隐私下学习阈值的近似最优样本复杂度。
• 提出一种新的私有内部点查找技术，以实现域大小的高效递归缩减。
• 将先前的上界Õ(2^{log* |X|})显著降低至Õ((log* |X|)^1.5)。
• 在准确率、隐私参数和域大小方面，为所提算法建立紧致的效用与隐私保障。

提出的方法

• 提出一种称为HeavyPaths的递归算法，结合StoppingPoint、OneRandomPath和LevelUp，以私有方式识别数据中的内部点。
• 使用输入相关的哈希函数将原始数据库嵌入更小的域中，使每级递归的域大小对数级减小。
• 采用基于深度的路径构造机制，确保所选点为内部点的概率较高。
• 应用高级差分隐私组合定理，以限制多个递归步骤和子程序中的隐私损失。
• 采用多级缩减策略，每一级均维护对内部点深度的私有估计，从而实现递归精炼。
• 引入基于统计检验的停止准则，以确定适当的递归深度，确保效用的同时保持隐私。

实验结果

研究问题

• RQ1 在近似差分隐私下，差分隐私学习阈值函数的最紧可能样本复杂度是多少？
• RQ2 Ω(log* |X|)的下界与Õ(2^{log* |X|})的上界之间的差距能否被显著缩小？
• RQ3 是否可能设计一种私有算法，实现在保持常数准确率和置信度的前提下，对log* |X|呈现次指数依赖？
• RQ4 如何利用通过私有哈希实现的递归域缩减来构建具有强效用保障的私有内部点查找器？
• RQ5 能否对隐私分析进行精细化处理，以在log* |X|和n方面获得ε和δ的更紧边界？

主要发现

• 该论文在差分隐私阈值学习的样本复杂度上实现了新的上界Õ((log* |X|)^1.5)，显著优于先前的Õ(2^{log* |X|})上界。
• 当n ≥ Ω((log* |X|)/ε) · log(1/(δ · log* |X|))时，算法HeavyPaths以至少1 - O(δ · log* |X|)的概率返回输入数据库的内部点。
• 该算法为(O(ε·√(log* |X|·log(1/(δ·log* |X|)))) + ε²·log* |X| + ε·log n, O(δ·(n + log* |X|)·e^{O(ε·√(log* |X|·log(1/(δ·log* |X|))))}))-差分隐私。
• 该方法通过私有哈希在每一级递归中对数级缩减域大小，从而实现高效且私有的路径构造。
• 分析表明，该算法在所有递归层级均保持高效率，确保最终输出为有效内部点的概率较高。
• 结果几乎确定了私有阈值学习的样本复杂度，弥合了长期存在的已知下界与上界之间的指数级差距。