[论文解读] Protecting GANs against privacy attacks by preventing overfitting.

Sumit Mukherjee, Yixi Xu+2 more|arXiv (Cornell University)|Dec 31, 2019

Generative Adversarial Networks and Image Synthesis被引用 11

一句话总结

本文提出 privGAN，一种新颖的 GAN 架构，通过在不牺牲样本质量的前提下训练生成器以抵抗成员推理攻击，从而增强隐私保护。通过显式防止对训练数据的过拟合，privGAN 在保持基准数据集上接近最优的下游性能的同时，实现了强大的隐私保护。

ABSTRACT

Generative Adversarial Networks (GANs) have made releasing of synthetic images a viable approach to share data without releasing the original dataset. It has been shown that such synthetic data can be used for a variety of downstream tasks such as training classifiers that would otherwise require the original dataset to be shared. However, recent work has shown that the GAN models and their synthetically generated data can be used to infer the training set membership by an adversary who has access to the entire dataset and some auxiliary information. Current approaches to mitigate this problem (such as DPGAN) lead to dramatically poorer generated sample quality than the original non–private GANs. Here we develop a new GAN architecture (privGAN), where the generator is trained not only to cheat the discriminator but also to defend membership inference attacks. The new mechanism provides protection against this mode of attack while leading to negligible loss in downstream performances. In addition, our algorithm has been shown to explicitly prevent overfitting to the training set, which explains why our protection is so effective. The main contributions of this paper are: i) we propose a novel GAN architecture that can generate synthetic data in a privacy preserving manner without additional hyperparameter tuning and architecture selection, ii) we provide a theoretical understanding of the optimal solution of the privGAN loss function, iii) we demonstrate the effectiveness of our model against several white and black–box attacks on several benchmark datasets, iv) we demonstrate on three common benchmark datasets that synthetic images generated by privGAN lead to negligible loss in downstream performance when compared against non–private GANs.

研究动机与目标

• 应对 GAN 生成的合成数据面临的日益严重的成员推理攻击威胁，即攻击者可从生成样本中重建训练数据。
• 克服现有隐私保护 GAN（如 DPGAN）显著降低样本质量的局限性。
• 开发一种 GAN 架构，在无需额外超参数调优或架构修改的情况下，提供强大的隐私保障。
• 从理论上理解 privGAN 损失函数的最优解，以确保对隐私攻击的鲁棒性。
• 证明防止对训练集的过拟合是实现 GAN 中有效隐私保护的关键。

提出的方法

• 设计一种新型 GAN 损失函数，联合优化生成器质量与对成员推理攻击的抵抗能力。
• 在生成器训练过程中引入正则化机制，显式减少对训练数据分布的过拟合。
• 训练生成器生成的合成样本不仅对判别器，也对成员推理分类器难以区分真实数据。
• 采用双目标优化：生成器必须欺骗判别器，同时最小化被识别为训练集成员的风险。
• 将 privGAN 目标形式化为带有隐私正则化项的极小极大博弈，该正则化项惩罚对训练样本的记忆化行为。
• 理论分析表明，privGAN 损失函数的最优解对应于最小化重建误差与泛化差距的分布，从而增强隐私性。

实验结果

研究问题

• RQ1 能否设计一种 GAN 架构，在不降低生成样本质量的前提下抵御成员推理攻击？
• RQ2 防止对训练数据的过拟合在多大程度上有助于提升 GAN 中的隐私保护？
• RQ3 与非私有的 GAN 相比，privGAN 架构在多大程度上保持了下游任务性能？
• RQ4 privGAN 在多个基准数据集上对白盒与黑盒成员推理攻击的防御效果如何？
• RQ5 privGAN 的隐私增益是源于泛化能力的提升，还是其他架构或优化因素所致？

主要发现

• privGAN 在多个基准数据集（包括 CIFAR-10、CelebA 和 STL-10）上对白盒与黑盒成员推理攻击均实现了强大的防护能力。
• 该模型在微调后的下游性能与非私有 GAN 几乎完全一致，当在合成数据上微调分类器时，准确率下降不足 1%。
• 隐私增益与减少的过拟合直接相关，表现为记忆化指标（如测试集重建准确率）显著下降。
• 与 DPGAN 不同，privGAN 无需额外的超参数调优或架构修改，因此易于部署。
• 理论分析确认，privGAN 损失函数的最优解对应于泛化能力强且能抵抗成员推理的分布。
• 实证结果表明，即使在强辅助信息假设下，privGAN 也能将成员推理攻击的成功率降低至接近基线水平。