[Paper Review] Fairwashing: the risk of rationalization

A\"ivodji, Ulrich, Arai, Hiromi+4 more|arXiv (Cornell University)|Jan 28, 2019

Explainable Artificial Intelligence (XAI)35 citations

TL;DR

This paper introduces 'fairwashing'—the deceptive use of model explanations to falsely imply fairness in inherently unfair black-box machine learning models. The authors propose LaundryML, a regularized rule list enumeration algorithm that generates fairer, high-fidelity surrogate models from unfair black-box models, demonstrating through empirical evaluation that such rationalized explanations can significantly reduce unfairness while preserving fidelity to the original model.

ABSTRACT

Black-box explanation is the problem of explaining how a machine learning model -- whose internal logic is hidden to the auditor and generally complex -- produces its outcomes. Current approaches for solving this problem include model explanation, outcome explanation as well as model inspection. While these techniques can be beneficial by providing interpretability, they can be used in a negative manner to perform fairwashing, which we define as promoting the false perception that a machine learning model respects some ethical values. In particular, we demonstrate that it is possible to systematically rationalize decisions taken by an unfair black-box model using the model explanation as well as the outcome explanation approaches with a given fairness metric. Our solution, LaundryML, is based on a regularized rule list enumeration algorithm whose objective is to search for fair rule lists approximating an unfair black-box model. We empirically evaluate our rationalization technique on black-box models trained on real-world datasets and show that one can obtain rule lists with high fidelity to the black-box model while being considerably less unfair at the same time.

Motivation & Objective

• To expose the risk of 'fairwashing'—the deceptive use of model explanations to create the false perception that an unfair model is ethical.
• To demonstrate that black-box explanations can be systematically manipulated to rationalize unfair decisions as fair.
• To develop a generic, fairness-constrained method for generating interpretable, high-fidelity surrogate models from unfair black-box models.
• To empirically validate that such rationalized explanations can achieve significantly lower unfairness than the original model while maintaining high fidelity.
• To raise awareness in the ML community about the ethical risks of relying on post-hoc explanations without verifying their fairness.

Proposed method

• Proposes LaundryML, a regularized rule list enumeration algorithm that optimizes for both fidelity to the black-box model and fairness according to a predefined metric.
• Uses a dual-objective optimization: maximize fidelity to the black-box model and minimize unfairness, controlled by a fairness regularization parameter β.
• Applies the algorithm to generate global (model explanation) and local (outcome explanation) surrogate models, enabling rationalization at both levels.
• Employs a greedy, iterative search over rule lists, pruning and refining candidates based on fidelity and fairness constraints.
• Supports both global and local explanation rationalization by adapting the optimization to either the full dataset or individual instances.
• Is generic and agnostic to the underlying black-box model and fairness metric, enabling broad applicability.

Experimental results

Research questions

• RQ1 Can post-hoc explanations be systematically manipulated to create the false perception of fairness in an inherently unfair black-box model?
• RQ2 To what extent can a surrogate model be both highly faithful to an unfair black-box model and significantly fairer than it?
• RQ3 Is it possible to generate interpretable rule lists that rationalize unfair decisions as fair, using only black-box access?
• RQ4 How does the trade-off between fidelity and fairness manifest in the rationalization process?
• RQ5 Can such rationalized explanations be detected as deceptive, or are they indistinguishable from genuine fairness?

Key findings

• On the Adult Income dataset, the best rationalized model achieved a fidelity of 0.908 and unfairness of 0.058, reducing unfairness by more than half compared to the original black-box model.
• On the ProPublica Recidivism dataset, the best model achieved a fidelity of 0.748 and unfairness of 0.080, again showing a substantial reduction in unfairness.
• For local outcome explanations, with β = 0.9, a completely fair model (unfairness = 0.0) was found to explain the decision for every rejected user in the minority group of the Adult Income dataset.
• In both datasets, the relative feature importance rankings from FairML showed a significant shift: sensitive attributes (e.g., gender, race) dropped from top positions to near the bottom in the rationalized models.
• The proportion of minority group users for whom a rationalized model agreed with the black-box decision reached 100% at β = 0.9, indicating strong coverage and consistency.
• The rationalized models were specifically tailored to the 'suing group', suggesting that fidelity may degrade on new, unseen minority members, highlighting a potential detection challenge.