[论文解读] Dikaios: Privacy Auditing of Algorithmic Fairness via Attribute Inference Attacks

Jan Aalmoes, Vasisht Duddu+1 more|arXiv (Cornell University)|Feb 4, 2022

Privacy-Preserving Technologies in Data被引用 5

一句话总结

Dikaios 是一种隐私审计工具，采用基于自适应阈值的属性推断攻击，评估机器学习中处理公平性算法的隐私风险。研究发现，公平性机制可能在类别不平衡的情况下不可预测地增加属性泄露，从而削弱其在可信人工智能设计中的可靠性。

ABSTRACT

Machine learning (ML) models have been deployed for high-stakes applications. Due to class imbalance in the sensitive attribute observed in the datasets, ML models are unfair on minority subgroups identified by a sensitive attribute, such as race and sex. In-processing fairness algorithms ensure model predictions are independent of sensitive attribute. Furthermore, ML models are vulnerable to attribute inference attacks where an adversary can identify the values of sensitive attribute by exploiting their distinguishable model predictions. Despite privacy and fairness being important pillars of trustworthy ML, the privacy risk introduced by fairness algorithms with respect to attribute leakage has not been studied. We identify attribute inference attacks as an effective measure for auditing blackbox fairness algorithms to enable model builder to account for privacy and fairness in the model design. We proposed Dikaios, a privacy auditing tool for fairness algorithms for model builders which leveraged a new effective attribute inference attack that account for the class imbalance in sensitive attributes through an adaptive prediction threshold. We evaluated Dikaios to perform a privacy audit of two in-processing fairness algorithms over five datasets. We show that our attribute inference attacks with adaptive prediction threshold significantly outperform prior attacks. We highlighted the limitations of in-processing fairness algorithms to ensure indistinguishable predictions across different values of sensitive attributes. Indeed, the attribute privacy risk of these in-processing fairness schemes is highly variable according to the proportion of the sensitive attributes in the dataset. This unpredictable effect of fairness mechanisms on the attribute privacy risk is an important limitation on their utilization which has to be accounted by the model builder.

研究动机与目标

• 为填补理解处理公平性算法如何影响属性隐私泄露的空白。
• 识别出由于敏感属性中的类别不平衡，公平性机制可能意外增加隐私风险。
• 为模型开发者开发一种实用的审计工具，以评估公平性算法的隐私影响。
• 证明现有属性推断攻击在现实世界数据不平衡条件下，不足以评估公平性算法的隐私影响。

提出的方法

• 在属性推断攻击中提出一种自适应预测阈值，以考虑敏感属性中的类别不平衡。
• 设计 Dikaios 作为一种黑盒审计工具，利用模型预测结果来推断敏感属性。
• 采用一种阈值自适应策略，通过与不平衡条件下模型预测分布对齐，提升推断准确性。
• 在五个现实世界数据集上对两种处理公平性算法进行攻击评估，以测量隐私泄露程度。
• 利用该攻击评估公平性算法是否真正实现了不同敏感属性群体之间的不可区分性。
• 将隐私审计整合到模型开发生命周期中，以支持隐私感知的公平性设计。

实验结果

研究问题

• RQ1 所提出的基于自适应阈值的属性推断攻击在暴露公平性感知模型中的隐私泄露方面效果如何？
• RQ2 敏感属性中的类别不平衡在多大程度上影响了处理公平性算法的隐私风险？
• RQ3 现有属性推断攻击能否可靠地审计黑盒模型中公平性机制的隐私影响？
• RQ4 不同数据集中敏感属性分布各异时，公平性算法的隐私风险如何变化？
• RQ5 在现实世界数据条件下，处理公平性算法在保护属性隐私方面存在哪些局限性？

主要发现

• 基于自适应阈值的属性推断攻击在类别不平衡条件下，显著优于以往的攻击方法，能更有效地推断敏感属性。
• 处理公平性算法无法确保在不同敏感属性群体之间实现预测的不可区分性，尤其在少数群体代表性不足时更为明显。
• 公平性算法引入的隐私风险高度可变且不可预测，取决于数据集中敏感属性的比例。
• 模型开发者无法可靠地假设公平性算法能保护属性隐私，因为同一算法在某些数据集中可能比在其他数据集中泄露更多敏感信息。
• 本研究揭示了公平性与隐私之间存在关键权衡，即公平性机制可能在无意中增加少数群体子群体的隐私泄露风险。
• Dikaios 使模型开发者能够主动检测并缓解隐私风险，从而在部署公平性感知模型前采取预防措施。