[论文解读] Provable trade-offs between private & robust machine learning.

Jamie Hayes|arXiv (Cornell University)|Jun 8, 2020

Adversarial Robustness in Machine Learning参考文献 50被引用 6

一句话总结

本文通过分析标准模型与鲁棒模型之间过拟合差异，建立了机器学习中隐私与鲁棒性之间可证明的权衡关系。研究发现，鲁棒模型可减少过拟合，从而降低隐私风险，尤其在训练数据充足时，挑战了鲁棒性必然损害隐私的假设。

ABSTRACT

Historically, machine learning methods have not been designed with security in mind. In turn, this has given rise to adversarial examples, carefully perturbed input samples aimed to mislead detection at test time, which have been applied to attack spam and malware classification, and more recently to attack image classification. Consequently, an abundance of research has been devoted to designing machine learning methods that are robust to adversarial examples. Unfortunately, there are desiderata besides robustness that a secure and safe machine learning model must satisfy, such as fairness and privacy. Recent work by Song et al. (2019) has shown, empirically, that there exists a trade-off between robust and private machine learning models. Models designed to be robust to adversarial examples often overfit on training data to a larger extent than standard (non-robust) models. If a dataset contains private information, then any statistical test that separates training and test data by observing a model's outputs can represent a privacy breach, and if a model overfits on training data, these statistical tests become easier. In this work, we identify settings where standard models will provably overfit to a larger extent in comparison to robust models, and as empirically observed in previous works, settings where the opposite behavior occurs. Thus, it is not necessarily the case that privacy must be sacrificed to achieve robustness. The degree of overfitting naturally depends on the amount of data available for training. We go on to formally characterize how the training set size factors into the privacy risks exposed by training a robust model. Finally, we empirically show our findings hold on image classification benchmark datasets, such as CIFAR-10.

研究动机与目标

• 探究鲁棒机器学习模型是否因过拟合增加而固有地损害数据隐私。
• 正式分析训练集大小如何影响鲁棒模型中的隐私风险。
• 识别鲁棒模型过拟合少于标准模型的条件，以挑战先前的实证观察。
• 为机器学习系统中观察到的隐私-鲁棒性权衡提供理论依据。
• 在CIFAR-10等基准数据集上实证验证理论发现。

提出的方法

• 使用泛化界对标准模型与鲁棒模型的泛化误差和过拟合进行理论分析。
• 形式化刻画训练集大小如何影响鲁棒模型中过拟合的程度。
• 推导鲁棒模型过拟合少于标准模型的条件。
• 使用统计检验建模通过模型输出泄露引发的隐私泄露，将过拟合与隐私风险关联。
• 在CIFAR-10上使用标准训练与鲁棒训练方法进行实证评估，比较过拟合与隐私暴露情况。
• 对不同训练集大小下的泛化误差与测试准确率进行定量比较。

实验结果

研究问题

• RQ1 在何种条件下，鲁棒模型的过拟合程度低于标准模型，从而降低隐私风险？
• RQ2 训练数据集的大小在多大程度上影响鲁棒机器学习模型相关的隐私风险？
• RQ3 是否存在可证明的模型鲁棒性与过拟合之间的关系，进而影响数据隐私？
• RQ4 在特定数据规模条件下，鲁棒模型是否能实现优于标准模型的隐私特性？
• RQ5 实证观察到的隐私-鲁棒性权衡在理论分析下在多大程度上成立？

主要发现

• 在某些数据环境下，鲁棒模型的过拟合程度可低于标准模型，与‘鲁棒性总是增加隐私风险’的假设相矛盾。
• 当训练数据充足时，鲁棒模型中的过拟合程度显著降低，从而减少隐私暴露。
• 训练集大小在决定隐私风险方面起着关键作用：更大的数据集可减少过拟合，从而降低成员推断攻击的可行性。
• 理论分析证实，当数据充足时，鲁棒模型的泛化能力可优于标准模型。
• 在CIFAR-10上的实证结果验证了理论发现，表明在大规模训练集下，鲁棒模型的过拟合程度更低。
• 模型过拟合带来的隐私风险并非与鲁棒性固有相关，而取决于数据可用性与模型容量。