[论文解读] See through Gradients: Image Batch Recovery via GradInversion

Hongxu Yin, Arun Mallya+4 more|arXiv (Cornell University)|Apr 15, 2021

Adversarial Robustness in Machine Learning参考文献 42被引用 30

一句话总结

GradInversion 从大规模网络和数据集中批量平均梯度中重建单个图像，使用标签恢复和组/现实性正则化，揭示基于梯度的学习中的隐私风险。

ABSTRACT

Training deep neural networks requires gradient estimation from data batches to update parameters. Gradients per parameter are averaged over a set of data and this has been presumed to be safe for privacy-preserving training in joint, collaborative, and federated learning applications. Prior work only showed the possibility of recovering input data given gradients under very restrictive conditions - a single input point, or a network with no non-linearities, or a small 32x32 px input batch. Therefore, averaging gradients over larger batches was thought to be safe. In this work, we introduce GradInversion, using which input images from a larger batch (8 - 48 images) can also be recovered for large networks such as ResNets (50 layers), on complex datasets such as ImageNet (1000 classes, 224x224 px). We formulate an optimization task that converts random noise into natural images, matching gradients while regularizing image fidelity. We also propose an algorithm for target class label recovery given gradients. We further propose a group consistency regularization framework, where multiple agents starting from different random seeds work together to find an enhanced reconstruction of original data batch. We show that gradients encode a surprisingly large amount of information, such that all the individual images can be recovered with high fidelity via GradInversion, even for complex datasets, deep networks, and large batch sizes.

研究动机与目标

• 通过展示 batch-gradient 泄漏，激发对基于梯度的协作和联邦学习的隐私风险。
• 开发 GradInversion，以从大型网络和数据集的平均批梯度中恢复原始图像。
• 提出通过正则化识别真实标签并提高重建保真度的技术。

提出的方法

• 将输入重建框定为最小化梯度-m 匹配损失再加上现实性正则化的优化。
• 通过利用最终全连接层的梯度来恢复批标签。
• 使用全变差、L2 和 BatchNorm 先验来实现保真性正则化，以生成逼真的图像。
• 通过联合优化多个种子并将输出注册以形成共识，引入分组一致性正则化。
• 引入 Langevin 风格的噪声以在重建过程中促进探索。

实验结果

研究问题

• RQ1 GradInversion 是否能够从大规模网络和高分辨率数据集中批量平均梯度中恢复原始图像？
• RQ2 从最终分类层的梯度中能够多准确地恢复批标签？
• RQ3 保真性和分组一致性正则化是否提高了恢复图像的真实感和保真度？
• RQ4 批量大小对重建质量和图像可辨识性有何影响？
• RQ5 该方法与先前的 ImageNet 规模任务梯度逆向方法相比有何不同？

主要发现

• GradInversion 可以从 ResNet-50 的梯度中对 48 的批量大小，逐步恢复详细的 224×224 ImageNet 图像。
• 从最终全连接层梯度中恢复标签极其有效，即使对于较大批量也能达到较高准确性。
• 保真性正则化（包括 BN 先验）显著提升了视觉质量和与原图的定量相似度。
• 通过多种种子优化和配准实现的分组一致性正则化进一步提升了重建保真度。
• GradInversion 在 ImageNet 梯度上的视觉和定量指标均优于先前方法。