[Paper Review] Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle

Luca Deck, Astrid Schomäcker+4 more|arXiv (Cornell University)|Apr 29, 2024

Impact of AI and Big Data on Business and SocietyDecision Sciences3 citations

TL;DR

This paper identifies eight distinct fairness desiderata in AI systems, maps them across the AI lifecycle stages, and analyzes how explainable AI (XAI) techniques can address each fairness goal at its most relevant lifecycle phase. The key contribution is a systematic framework linking specific fairness objectives to targeted XAI interventions throughout development, deployment, and monitoring.

ABSTRACT

The widespread use of artificial intelligence (AI) systems across various domains is increasingly surfacing issues related to algorithmic fairness, especially in high-stakes scenarios. Thus, critical considerations of how fairness in AI systems might be improved -- and what measures are available to aid this process -- are overdue. Many researchers and policymakers see explainable AI (XAI) as a promising way to increase fairness in AI systems. However, there is a wide variety of XAI methods and fairness conceptions expressing different desiderata, and the precise connections between XAI and fairness remain largely nebulous. Besides, different measures to increase algorithmic fairness might be applicable at different points throughout an AI system's lifecycle. Yet, there currently is no coherent mapping of fairness desiderata along the AI lifecycle. In this paper, we we distill eight fairness desiderata, map them along the AI lifecycle, and discuss how XAI could help address each of them. We hope to provide orientation for practical applications and to inspire XAI research specifically focused on these fairness desiderata.

Motivation & Objective

• To address the lack of a coherent mapping between fairness goals and the AI lifecycle.
• To clarify the diverse conceptions of fairness in AI and distinguish them as distinct desiderata.
• To examine how XAI techniques can be strategically applied to fulfill specific fairness objectives at different lifecycle stages.
• To provide a structured, interdisciplinary foundation for researchers, developers, and regulators to guide fairness-focused XAI development.
• To illustrate the framework’s utility through a case study on the COMPAS recidivism prediction system.

Proposed method

• The authors distill eight fairness desiderata from interdisciplinary literature: fairness understanding, data fairness, formal fairness, perceived fairness, fairness with human oversight, empowering fairness, long-term fairness, and informational fairness.
• They map each fairness desideratum to specific phases of the AI lifecycle—such as data collection, model training, deployment, and monitoring—based on when each fairness goal becomes most relevant.
• For each mapped desideratum, the paper analyzes which XAI methods (e.g., feature importance, counterfactual explanations, post-hoc interpretability) are most suitable for addressing it at that lifecycle stage.
• The framework is validated through a detailed case study on the COMPAS recidivism algorithm, demonstrating how XAI can be aligned with fairness goals across different lifecycle phases.
• The approach integrates insights from computer science, ethics, law, and social science to ensure conceptual robustness and interdisciplinary relevance.
• The authors do not propose a prescriptive process model but instead offer a conceptual roadmap for aligning XAI with fairness objectives based on lifecycle context.

Experimental results

Research questions

• RQ1 Which fairness desiderata are most relevant at different stages of the AI lifecycle?
• RQ2 How can XAI techniques be strategically selected to address specific fairness goals at their most appropriate lifecycle phase?
• RQ3 What are the limitations of current XAI approaches in addressing diverse fairness conceptions beyond technical fairness?
• RQ4 How does the COMPAS case illustrate the potential and challenges of aligning XAI with fairness across the lifecycle?
• RQ5 To what extent can this framework be generalized to low-risk and generative AI systems?

Key findings

• The study identifies eight distinct fairness desiderata that are conceptually distinct and not mutually exclusive, each requiring tailored interventions at different lifecycle stages.
• XAI is not a one-size-fits-all solution for fairness; its effectiveness depends on aligning specific XAI methods with specific fairness goals at the right lifecycle phase.
• The COMPAS case study demonstrates that XAI can help detect and explain biased outcomes, especially when used to support perceived fairness and fairness with human oversight.
• Long-term fairness and empowering fairness are most effectively addressed through XAI mechanisms that support ongoing monitoring and user agency, particularly in post-deployment phases.
• The framework reveals that current XAI research often overlooks non-technical fairness dimensions such as perceived fairness and informational fairness, which are critical for stakeholder trust.
• The approach highlights that generative AI systems, even if labeled low-risk, can still perpetuate representational harms and require fairness-aware XAI from the outset.