[论文解读] RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network

Chuan Ma, Jun Li+5 more|arXiv (Cornell University)|Jul 4, 2020

Privacy-Preserving Technologies in Data参考文献 27被引用 12

一句话总结

该论文提出RDP-GAN，一种生成对抗网络，通过在训练过程中向判别器的损失函数注入噪声，实现Rényi差分隐私（RDP）。通过在子采样和迭代训练下解析推导隐私损失，并引入自适应噪声调节机制，RDP-GAN在保持高质量生成结果的同时，相比基于梯度的DP-GAN显著提升了隐私保护能力。

ABSTRACT

Generative adversarial network (GAN) has attracted increasing attention recently owing to its impressive ability to generate realistic samples with high privacy protection. Without directly interactive with training examples, the generative model can be fully used to estimate the underlying distribution of an original dataset while the discriminative model can examine the quality of the generated samples by comparing the label values with the training examples. However, when GANs are applied on sensitive or private training examples, such as medical or financial records, it is still probable to divulge individuals' sensitive and private information. To mitigate this information leakage and construct a private GAN, in this work we propose a Rényi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random noises on the value of the loss function during training. Moreover, we derive the analytical results of the total privacy loss under the subsampling method and cumulated iterations, which show its effectiveness on the privacy budget allocation. In addition, in order to mitigate the negative impact brought by the injecting noise, we enhance the proposed algorithm by adding an adaptive noise tuning step, which will change the volume of added noise according to the testing accuracy. Through extensive experimental results, we verify that the proposed algorithm can achieve a better privacy level while producing high-quality samples compared with a benchmark DP-GAN scheme based on noise perturbation on training gradients.

研究动机与目标

• 解决在医疗或金融记录等敏感数据上训练的GAN所面临的隐私泄露风险。
• 通过采用Rényi差分隐私（RDP）而非传统差分隐私，实现强于标准DP-GAN的隐私保障。
• 在子采样和多次训练迭代下，推导出隐私预算的解析分配方法。
• 通过自适应噪声调节机制缓解噪声注入导致的性能下降。

提出的方法

• 在训练过程中直接向判别器的损失函数注入随机噪声，以实现Rényi差分隐私（RDP）。
• 利用RDP组合定理，推导出在子采样和累积训练迭代下的总隐私损失的解析表达式。
• 应用后处理定理，确保生成器继承来自差分私密判别器的隐私保证。
• 引入自适应噪声调节步骤，根据实时测试准确率动态调整噪声方差，以平衡隐私与生成质量。
• 采用RDP组合框架，通过邻近数据集下损失函数比值的RDP散度计算隐私边界。
• 使用一系列数学不等式和级数展开，对RDP散度进行有界处理，并推导出闭式隐私预算。

实验结果

研究问题

• RQ1 在判别器损失函数中注入噪声而非梯度，是否能在GAN中实现更好的隐私-效用权衡？
• RQ2 在GAN设置下，如何在子采样和多次训练迭代下解析计算总隐私损失？
• RQ3 自适应噪声调节对在保持高样本质量的同时维持隐私有何影响？
• RQ4 与基于梯度扰动的DP-GAN相比，RDP-GAN在隐私预算和样本保真度方面表现如何？

主要发现

• 通过使用RDP组合，RDP-GAN相比标准DP-GAN实现了更紧致的隐私预算边界，从而获得更优的隐私-效用权衡。
• 在子采样和迭代训练下对隐私损失的解析推导，使得隐私预算能够在训练步骤间实现精确分配。
• 自适应噪声调节机制有效减轻了噪声注入的负面影响，即使在强隐私约束下也能保持高质量的样本生成。
• 实验结果表明，RDP-GAN在隐私保护和样本质量方面均优于基准DP-GAN，尤其在高隐私约束下表现更优。
• 理论分析证实，生成器可通过后处理定理继承来自差分私密判别器的隐私保证。