[Paper Review] Characterizing the risk of fairwashing

Ulrich Aïvodji, Hiromi Arai+2 more|arXiv (Cornell University)|Jun 14, 2021

Law, Economics, and Judicial Systems7 citations

TL;DR

This paper investigates the risk of fairwashing—where unfair black-box models are rationalized through manipulated, fair-seeming explanations—by analyzing their fidelity-unfairness trade-offs. It demonstrates that fairwashed explanations generalize across groups and transfer across models, making detection extremely difficult, and proposes a risk quantification method using the Fairness In The Rashomon Set (FaiRS) framework.

ABSTRACT

Fairwashing refers to the risk that an unfair black-box model can be explained by a fairer model through post-hoc explanation manipulation. In this paper, we investigate the capability of fairwashing attacks by analyzing their fidelity-unfairness trade-offs. In particular, we show that fairwashed explanation models can generalize beyond the suing group (i.e., data points that are being explained), meaning that a fairwashed explainer can be used to rationalize subsequent unfair decisions of a black-box model. We also demonstrate that fairwashing attacks can transfer across black-box models, meaning that other black-box models can perform fairwashing without explicitly using their predictions. This generalization and transferability of fairwashing attacks imply that their detection will be difficult in practice. Finally, we propose an approach to quantify the risk of fairwashing, which is based on the computation of the range of the unfairness of high-fidelity explainers.

Motivation & Objective

• To investigate the manipulability of fairwashing attacks in post-hoc explanation systems.
• To assess whether fairwashed explanations can generalize beyond the targeted group (i.e., to other demographic groups).
• To evaluate whether fairwashing attacks can transfer across different black-box models without retraining.
• To propose a method for quantifying the risk of fairwashing using the Fairness In The Rashomon Set (FaiRS) framework.
• To challenge the viability of fidelity-based detection methods for fairwashing by showing their limitations.

Proposed method

• The authors use an ϵ-constraint method to solve a multi-objective optimization problem that balances explanation fidelity and fairness constraints.
• They train surrogate models (e.g., rule lists, logistic regression) to explain black-box models while enforcing fairness constraints (e.g., equal opportunity, statistical parity).
• Generalization is evaluated by measuring the fidelity of a fairwashed explanation on a non-targeted group, comparing it to fidelity on the original (suing) group.
• Transferability is assessed by evaluating the same fairwashed explanation on a different black-box model, measuring fidelity and unfairness on the new model.
• Risk quantification is performed using the FaiRS framework, which computes the range of unfairness across high-fidelity explanations in the Rashomon set.
• Experiments are conducted across multiple datasets (Adult Income, COMPAS, Default Credit, Marketing), black-box models (AdaBoost, DNN, RF, XGBoost), and fairness criteria (EOdds, EOpp, PE, SP).

Experimental results

Research questions

• RQ1 Can fairwashed explanations generalize beyond the group they were specifically designed to rationalize?
• RQ2 Can fairwashing attacks transfer from one black-box model to another, even without access to the new model’s predictions?
• RQ3 Is fidelity alone a reliable metric for detecting fairwashing attacks?
• RQ4 Can the Fairness In The Rashomon Set (FaiRS) framework effectively quantify the risk of fairwashing?
• RQ5 What is the range of unfairness in high-fidelity explanations, and how does it relate to manipulability?

Key findings

• Fairwashed explanations generalize beyond the suing group, with fidelity on non-targeted groups comparable to that on the original group, indicating broad rationalization capability.
• Fairwashing attacks transfer across black-box models, as the same explanation maintains high fidelity on different models, even when the models are not identical.
• The fidelity of fairwashed explanations remains high (e.g., >0.94) across diverse datasets and fairness constraints, suggesting strong manipulability.
• Fidelity-based detection of fairwashing is ineffective due to the high generalization and transferability of attacks, as shown by consistent high-fidelity scores across groups and models.
• The FaiRS framework successfully quantifies the risk of fairwashing by revealing the range of unfairness in high-fidelity explanations, offering a robust alternative to fidelity-only metrics.
• For the Marketing dataset with ϵ=0.03, fidelity remained above 0.94 across all models and fairness criteria, with unfairness values consistently below 0.03, indicating strong attack resilience.