[Paper Review] Interpretable Data-Based Explanations for Fairness Debugging

Romila Pradhan, Jiongli Zhu+2 more|arXiv (Cornell University)|Dec 17, 2021

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

Gopher introduces a novel framework for debugging fairness issues in machine learning by identifying compact, interpretable, and causally responsible training data patterns that explain model bias. It uses causal responsibility to rank data subsets whose removal or update would most reduce bias, enabling developers to trace and mitigate bias at its root source in training data.

ABSTRACT

A wide variety of fairness metrics and eXplainable Artificial Intelligence (XAI) approaches have been proposed in the literature to identify bias in machine learning models that are used in critical real-life contexts. However, merely reporting on a model's bias, or generating explanations using existing XAI techniques is insufficient to locate and eventually mitigate sources of bias. We introduce Gopher, a system that produces compact, interpretable and causal explanations for bias or unexpected model behavior by identifying coherent subsets of the training data that are root-causes for this behavior. Specifically, we introduce the concept of causal responsibility that quantifies the extent to which intervening on training data by removing or updating subsets of it can resolve the bias. Building on this concept, we develop an efficient approach for generating the top-k patterns that explain model bias that utilizes techniques from the machine learning (ML) community to approximate causal responsibility and uses pruning rules to manage the large search space for patterns. Our experimental evaluation demonstrates the effectiveness of Gopher in generating interpretable explanations for identifying and debugging sources of bias.

Motivation & Objective

• To address the gap in existing XAI methods that cannot trace bias back to training data sources.
• To develop a system that generates interpretable, compact, and causal explanations for unexpected or discriminatory model behavior.
• To enable ML practitioners to debug and mitigate bias by identifying specific training data subsets responsible for unfair outcomes.
• To formalize the concept of causal responsibility for training data subsets in fairness debugging.
• To provide a scalable method for identifying top-k data patterns that, when modified, reduce model bias the most.

Proposed method

• Introduces the concept of causal responsibility to quantify how much removing or updating a training data subset reduces model bias.
• Develops an efficient algorithm to approximate causal responsibility using techniques from machine learning and pruning rules to manage the large search space of potential data patterns.
• Uses pattern mining to discover cohesive, interpretable subsets of training data that are causally linked to bias.
• Employs a top-k selection strategy to return the most influential data patterns explaining model bias.
• Leverages techniques from database provenance and machine unlearning for potential future integration to improve efficiency.
• Applies pruning heuristics to reduce computational cost while maintaining accuracy in identifying responsible data subsets.

Experimental results

Research questions

• RQ1 What training data patterns are causally responsible for unfair model behavior?
• RQ2 How can we efficiently identify the top-k data subsets whose removal or update would most reduce model bias?
• RQ3 Can we generate interpretable, compact, and causally grounded explanations for bias that trace back to specific training data?
• RQ4 How does causal responsibility compare to correlation-based or feature-based explanations in identifying root causes of bias?
• RQ5 Can the framework be extended to non-differentiable models like tree-based or clustering algorithms?

Key findings

• Gopher successfully identifies interpretable data patterns that explain model bias, such as 'unmarried females who are homeowners' as a root cause of gender bias in loan prediction.
• The system efficiently computes causal responsibility approximations, enabling scalable discovery of top-k responsible data subsets.
• Experimental evaluation confirms that Gopher’s explanations correctly identify known sources of bias in datasets with well-understood ground-truth biases.
• Gopher’s explanations outperform feature-based and instance-based XAI methods in diagnosing the root cause of bias in training data.
• The framework is effective in detecting and explaining bias due to data imbalance, misclassification, and measurement errors in training data.
• The approach is generalizable and can be extended to support a wide range of ML models beyond differentiable ones.