[Paper Review] AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias

Rachel Bellamy, Kuntal Dey+16 more|arXiv (Cornell University)|Oct 3, 2018

Ethics and Social Impacts of AI18 references265 citations

TL;DR

Introduces AI Fairness 360 (AIF360), an open-source Python toolkit that provides bias detection metrics, explanations, and mitigation algorithms, plus an interactive web UI for industrial usability and benchmarking.

ABSTRACT

Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This paper introduces a new open source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license {https://github.com/ibm/aif360). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms to use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience (https://aif360.mybluemix.net) that provides a gentle introduction to the concepts and capabilities for line-of-business users, as well as extensive documentation, usage guidance, and industry-specific tutorials to enable data scientists and practitioners to incorporate the most appropriate tool for their problem into their work products. The architecture of the package has been engineered to conform to a standard paradigm used in data science, thereby further improving usability for practitioners. Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.

Motivation & Objective

• Promote understanding of fairness metrics and mitigation techniques in ML.
• Provide an open, extensible platform for researchers and industry to share and benchmark fairness algorithms.
• Ease the transition of fairness research into industrial practice through usable tooling and documentation.

Proposed method

• Define an extensible architecture with dataset representations, metrics, explainers, and bias-mitigation algorithms.
• Incorporate 71+ bias detection metrics, 9 bias mitigation methods, and a metric explainer framework.
• Provide a standard pipeline (dataset -> fair dataset -> model -> predictions) to evaluate bias at multiple stages.
• Develop a web-based interactive experience and extensive documentation to aid practitioners.
• Implement a rigorous testing and continuous integration setup to maintain code quality.

Experimental results

Research questions

• RQ1 How can a unified open-source toolkit support the detection, understanding, and mitigation of algorithmic bias across diverse datasets and models?
• RQ2 What metrics and mitigation strategies are most effective for different fairness definitions and deployment contexts?
• RQ3 How can explanations and localization of bias (in protected attributes and features) aid users in selecting appropriate interventions?

Key findings

• AIF360 integrates bias metrics, mitigation algorithms, and explanations in a single open-source package to facilitate benchmarking and adoption.
• Pre-processing and in-processing approaches (e.g., Reweighing, Optimized Pre-processing, Adversarial Debiasing) generally improve fairness metrics with varying impact on accuracy across datasets.
• Post-processing methods (e.g., Equalized Odds, Calibrated Equalized Odds, Reject Option) offer alternatives when model retraining is not possible, with trade-offs in accuracy and fairness.
• An interactive web experience and extensive tutorials support business users, developers, and researchers in applying the toolkit to real-world problems.