[论文解读] DTGAN: Differential Private Training for Tabular GANs

Aditya Kunar, Robert Birke+2 more|arXiv (Cornell University)|Jul 6, 2021

Privacy-Preserving Technologies in Data参考文献 25被引用 9

一句话总结

DTGAN 提出了一种用于表格数据合成的差异化私有条件Wasserstein GAN，通过在生成器或判别器上使用DP-SGD进行训练。实验表明，在严格隐私预算 ε=1 下，对判别器应用DP-SGD（DTGAN D）在4个机器学习模型上的平均精确率提高了18%，同时将成员推断攻击的成功率有效控制在约50%。

ABSTRACT

Tabular generative adversarial networks (TGAN) have recently emerged to cater to the need of synthesizing tabular data -- the most widely used data format. While synthetic tabular data offers the advantage of complying with privacy regulations, there still exists a risk of privacy leakage via inference attacks due to interpolating the properties of real data during training. Differential private (DP) training algorithms provide theoretical guarantees for training machine learning models by injecting statistical noise to prevent privacy leaks. However, the challenges of applying DP on TGAN are to determine the most optimal framework (i.e., PATE/DP-SGD) and neural network (i.e., Generator/Discriminator)to inject noise such that the data utility is well maintained under a given privacy guarantee. In this paper, we propose DTGAN, a novel conditional Wasserstein tabular GAN that comes in two variants DTGAN_G and DTGAN_D, for providing a detailed comparison of tabular GANs trained using DP-SGD for the generator vs discriminator, respectively. We elicit the privacy analysis associated with training the generator with complex loss functions (i.e., classification and information losses) needed for high quality tabular data synthesis. Additionally, we rigorously evaluate the theoretical privacy guarantees offered by DP empirically against membership and attribute inference attacks. Our results on 3 datasets show that the DP-SGD framework is superior to PATE and that a DP discriminator is more optimal for training convergence. Thus, we find (i) DTGAN_D is capable of maintaining the highest data utility across 4 ML models by up to 18% in terms of the average precision score for a strict privacy budget, epsilon = 1, as compared to the prior studies and (ii) DP effectively prevents privacy loss against inference attacks by restricting the success probability of membership attacks to be close to 50%.

研究动机与目标

• 解决表格GAN因成员推断和属性推断攻击导致的隐私泄露风险。
• 评估在表格GAN中应用差分隐私（DP）的最佳位置——是应用于生成器还是判别器。
• 比较DP-SGD与PATE在训练隐私保护型表格GAN方面的有效性。
• 评估DP正则化GAN在实际场景中对真实推断攻击的鲁棒性。
• 量化在DP训练中复杂生成器损失（分类损失和信息损失）带来的隐私成本。

提出的方法

• 提出DTGAN，一种带有梯度惩罚的条件Wasserstein GAN，用于实现高质量的表格数据合成。
• 引入两种变体：DTGAN D（在判别器上应用DP-SGD）和DTGAN G（在生成器上应用DP-SGD）。
• 采用Rényi差分隐私（RDP）分析，以考虑附加生成器损失（分类损失和信息损失）带来的隐私成本。
• 在两种变体中均采用子采样策略，以减少训练过程中的隐私预算消耗。
• 在反向传播过程中通过噪声注入使用DP-SGD，以确保理论上的隐私保障。
• 在评估中采用朴素和基于相关性的特征提取方法，用于成员推断和属性推断攻击。

实验结果

研究问题

• RQ1 在严格隐私预算下，DP-SGD与PATE框架中，哪一个在训练表格GAN时更有效？
• RQ2 在表格GAN中，将DP-SGD应用于生成器还是判别器，能在隐私与数据效用之间实现更优平衡？
• RQ3 在DP训练中，引入复杂生成器损失（分类损失和信息损失）如何影响整体隐私预算？
• RQ4 DP正则化的表格GAN在实践中对成员推断和属性推断攻击的抵抗能力如何？
• RQ5 在不同DP训练配置下，表格GAN的数据效用与隐私之间的权衡关系如何？

主要发现

• 在所有评估指标下，DP-SGD在数据效用方面优于PATE框架，DTGAN D在ε=1下相比先前工作实现了18%更高的平均精确率得分。
• DTGAN D（在判别器上应用DP-SGD）在整体数据效用方面表现最佳，且在训练收敛性方面优于DTGAN G。
• 对DTGAN D和DTGAN G的成员推断攻击成功率降低至约50%，表明具备强大的隐私保护能力。
• 尽管效用更高，DTGAN G在属性推断攻击中的鲁棒性略低于PATE-GAN和DP-WGAN，凸显了效用与隐私之间的权衡。
• 所有DP基线模型（包括DTGAN）均将成员攻击成功率降低至接近随机水平（约50%），证实了理论隐私保障的有效性。
• 通过Rényi DP精确量化了附加生成器损失（分类损失和信息损失）的隐私成本，实现了精确的预算核算。