[论文解读] A Framework for Fairness: A Systematic Review of Existing Fair AI Solutions

Brianna Richardson, Juan E. Gilbert|arXiv (Cornell University)|Dec 10, 2021

Ethics and Social Impacts of AI被引用 30

一句话总结

本研究系统性地回顾算法偏见，调查公平性解决方案空间（工具包和清单），分析实际不足，并提出将研究者与从业者联系起来的建议。

ABSTRACT

In a world of daily emerging scientific inquisition and discovery, the prolific launch of machine learning across industries comes to little surprise for those familiar with the potential of ML. Neither so should the congruent expansion of ethics-focused research that emerged as a response to issues of bias and unfairness that stemmed from those very same applications. Fairness research, which focuses on techniques to combat algorithmic bias, is now more supported than ever before. A large portion of fairness research has gone to producing tools that machine learning practitioners can use to audit for bias while designing their algorithms. Nonetheless, there is a lack of application of these fairness solutions in practice. This systematic review provides an in-depth summary of the algorithmic bias issues that have been defined and the fairness solution space that has been proposed. Moreover, this review provides an in-depth breakdown of the caveats to the solution space that have arisen since their release and a taxonomy of needs that have been proposed by machine learning practitioners, fairness researchers, and institutional stakeholders. These needs have been organized and addressed to the parties most influential to their implementation, which includes fairness researchers, organizations that produce ML algorithms, and the machine learning practitioners themselves. These findings can be used in the future to bridge the gap between practitioners and fairness experts and inform the creation of usable fair ML toolkits.

研究动机与目标

• 在ML流程中定义并对算法偏差源进行分类（预先存在的、技术性、新兴的）
• 调查现有的公平性解决方案空间，包括软件工具包和清单。
• 评估在实际中公平性工具的使用情况并识别注意事项与设计缺陷。
• 提供将公平性研究与行业需求及从业者实践对齐的建议。

提出的方法

• 文献综合与偏差类型的分类学发展（预先存在、技术性、新兴）
• 编目并描述来自产业界和学术界的主要公平性工具包和清单。
• 分析公平性解决方案的可行性、易用性和设计空白。
• 讨论警示，如度量冲突、伦理洗白，以及社会-技术因素等。
• 提出面向公平性研究者、组织和ML从业者的可行动性建议。

实验结果

研究问题

• RQ1 驱动公平性关注的算法偏差的主要入口点和形式有哪些？
• RQ2 存在哪些公平性工具包和清单，它们提供了哪些特征或缺乏哪些特征？
• RQ3 当前的公平性解决方案在实践中的表现如何，它们存在哪些注意事项？
• RQ4 哪些建议可以弥合公平性研究与行业实践之间的差距？

主要发现

• 确立了偏差的分类法：预先存在的、技术性，以及新兴的（部署）偏差。
• 记录了一个多样化的解决方案空间，由软件工具包和生命周期清单主导。
• 强调在真实应用中公平性研究者与ML从业者之间的差距。
• 指出如公平性度量冲突、鲁棒性问题，以及伦理/技术平衡等挑战。
• 建议以人为本的设计以及面向领域的、可用的公平AI工具包以提升采用率。