[Paper Review] Aequitas: A Bias and Fairness Audit Toolkit

Pedro Saleiro, Benedict Kuester+6 more|arXiv (Cornell University)|Nov 14, 2018

Ethics and Social Impacts of AISocial Sciences27 references64 citations

TL;DR

Aequitas is an open-source toolkit released in 2018 that enables auditing of ML models for multiple bias and fairness metrics across population sub-groups, integrated as a Python library, CLI, and web app to support policy-relevant decisions.

ABSTRACT

Recent work has raised concerns on the risk of unintended bias in AI systems being used nowadays that can affect individuals unfairly based on race, gender or religion, among other possible characteristics. While a lot of bias metrics and fairness definitions have been proposed in recent years, there is no consensus on which metric/definition should be used and there are very few available resources to operationalize them. Therefore, despite recent awareness, auditing for bias and fairness when developing and deploying AI systems is not yet a standard practice. We present Aequitas, an open source bias and fairness audit toolkit that is an intuitive and easy to use addition to the machine learning workflow, enabling users to seamlessly test models for several bias and fairness metrics in relation to multiple population sub-groups. Aequitas facilitates informed and equitable decisions around developing and deploying algorithmic decision making systems for both data scientists, machine learning researchers and policymakers.

Motivation & Objective

• Motivate the need for practical bias and fairness auditing in AI systems impacting public policy.
• Provide an operational toolkit that computes diverse bias and fairness metrics across protected groups.
• Bridge data science and policy by delivering a user-friendly interface for both technical and non-technical users.
• Promote standard auditing practices to inform model selection, deployment, and periodic re-evaluation in production.

Proposed method

• Define a wide set of group-based metrics for distribution, error-based, and impact-oriented fairness.
• Allow comparison across multiple protected attribute groups using a reference group and disparity measures.
• Incorporate a tunable parity parameter tau to control acceptable disparity ranges (e.g., the 80% rule).
• Support pre-deployment, holdout, and post-deployment audits, including in A/B test settings.
• Provide a user-friendly Fairness Tree to guide metric selection based on the policy context and intervention costs.
• Deliver outputs via Python library, CLI, and a tailored web app for policymakers.

Experimental results

Research questions

• RQ1 How can we operationalize and compare multiple bias and fairness metrics across several demographic groups in public-policy ML applications?
• RQ2 What disparity measures (and reference-group choices) best reflect fairness concerns for different intervention contexts (assistive vs punitive)?
• RQ3 Can a practical toolkit help bridge data scientists and policymakers to adopt routine bias audits in real-world deployments?
• RQ4 How do different policy domains (criminal justice, health, public safety) exhibit distinct fairness challenges when applying ML risk scores?

Key findings

• Aequitas enables auditing of bias and fairness metrics across multiple population sub-groups during model development and deployment.
• The toolkit implements distributional, error-based, and impact-based fairness metrics with pairwise group disparity comparisons against a reference group.
• A tunable tau parameter provides flexible fairness constraints, generalizing notions like the 80% rule.
• Case studies across criminal justice, public health, and policing demonstrate detectable biases and the comparative impact of different models and baselines.
• Audits revealed pragmatic differences between expert baselines and ML models, with ML models often reducing certain biases compared with expert heuristics in some cases.
• The Fairness Tree aids non-technical policymakers in selecting relevant fairness metrics aligned with intervention goals.