[Paper Review] Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models

Zhibo Wang, Xiaowei Dong+5 more|arXiv (Cornell University)|Mar 3, 2022

Ethics and Social Impacts of AI4 citations

TL;DR

This paper proposes Fairness-aware Adversarial Perturbation (FAAP), a novel method to mitigate bias in deployed deep learning models without modifying their architecture or parameters. FAAP uses an adversarial generator to perturb input data such that protected attributes (e.g., gender, ethnicity) become indistinguishable in the model's latent space, thereby reducing unfair predictions. The approach achieves up to 66.1% reduction in demographic parity difference and 61.9% improvement in equal opportunity difference on commercial APIs with minimal accuracy drop.

ABSTRACT

Prioritizing fairness is of central importance in artificial intelligence (AI) systems, especially for those societal applications, e.g., hiring systems should recommend applicants equally from different demographic groups, and risk assessment systems must eliminate racism in criminal justice. Existing efforts towards the ethical development of AI systems have leveraged data science to mitigate biases in the training set or introduced fairness principles into the training process. For a deployed AI system, however, it may not allow for retraining or tuning in practice. By contrast, we propose a more flexible approach, i.e., fairness-aware adversarial perturbation (FAAP), which learns to perturb input data to blind deployed models on fairness-related features, e.g., gender and ethnicity. The key advantage is that FAAP does not modify deployed models in terms of parameters and structures. To achieve this, we design a discriminator to distinguish fairness-related attributes based on latent representations from deployed models. Meanwhile, a perturbation generator is trained against the discriminator, such that no fairness-related features could be extracted from perturbed inputs. Exhaustive experimental evaluation demonstrates the effectiveness and superior performance of the proposed FAAP. In addition, FAAP is validated on real-world commercial deployments (inaccessible to model parameters), which shows the transferability of FAAP, foreseeing the potential of black-box adaptation.

Motivation & Objective

• To address fairness issues in deployed deep learning models where retraining or structural changes are infeasible.
• To develop a method that improves fairness at inference time without access to model parameters or architecture.
• To enable black-box fairness mitigation by perturbing inputs to hide protected attributes from model representations.
• To achieve robust fairness improvements while preserving model accuracy and generalization.

Proposed method

• A generator network is trained to create adversarial perturbations that obscure protected attributes in input data.
• A discriminator is trained to classify protected attributes (e.g., gender) from the latent representations of the deployed model.
• The generator and discriminator are trained in an adversarial manner, where the generator aims to fool the discriminator into failing to detect protected attributes.
• The method leverages latent space representations from the deployed model to guide perturbation generation, ensuring perturbations are effective in hiding fairness-related features.
• Transferability is enabled by using surrogate models (e.g., normal or fair-trained models) to train the generator when the target model is black-box.
• The approach is evaluated using fairness metrics like demographic parity (DP) and equal opportunity difference (DEO) on both synthetic and real-world commercial APIs.

Experimental results

Research questions

• RQ1 Can adversarial perturbations be used to mitigate fairness bias in deployed deep learning models without modifying their parameters or architecture?
• RQ2 How effective is the proposed method in reducing demographic parity and equal opportunity differences in real-world black-box models?
• RQ3 To what extent can perturbations trained on surrogate models transfer to commercial, inaccessible APIs?
• RQ4 Does the method preserve model accuracy while significantly improving fairness metrics?
• RQ5 Can the perturbations effectively decorrelate protected attributes from model predictions in the latent space?

Key findings

• FAAP improved demographic parity (DP) by 27.5% and reduced equal opportunity difference (DEO) by 66.1% on average for normal training models, with only 1.5% accuracy degradation.
• On Alibaba’s commercial API, FAAP reduced DEO from 0.0952 to 0.0368 and improved DP from 0.1768 to 0.1475, with a 0.26% accuracy drop.
• On Baidu’s commercial API, FAAP achieved a 41.1% improvement in DP and a 64.8% reduction in DEO, with a 2.89% accuracy drop.
• T-SNE visualization confirmed that FAAP perturbations mix representations of different protected attributes (e.g., male/female) in the latent space, reducing separability.
• The method demonstrated strong transferability to real-world black-box APIs, indicating potential for practical deployment in production systems.
• The approach effectively decorrelates protected attributes from predictions while maintaining model utility, as shown by consistent fairness gains across multiple datasets and models.