[Paper Review] On Discrimination Discovery and Removal in Ranked Data using Causal Graph

Yongkai Wu, Lu Zhang+1 more|arXiv (Cornell University)|Mar 5, 2018

Ethics and Social Impacts of AI29 references4 citations

TL;DR

This paper proposes a causal graph-based framework to detect and remove both direct and indirect discrimination in ranked data by mapping ranking positions to continuous qualification scores. It extends path-specific effect analysis to mixed-variable causal graphs, enabling accurate discrimination measurement in ranking systems—outperforming statistical parity methods that fail to capture causal biases.

ABSTRACT

Predictive models learned from historical data are widely used to help companies and organizations make decisions. However, they may digitally unfairly treat unwanted groups, raising concerns about fairness and discrimination. In this paper, we study the fairness-aware ranking problem which aims to discover discrimination in ranked datasets and reconstruct the fair ranking. Existing methods in fairness-aware ranking are mainly based on statistical parity that cannot measure the true discriminatory effect since discrimination is causal. On the other hand, existing methods in causal-based anti-discrimination learning focus on classification problems and cannot be directly applied to handle the ranked data. To address these limitations, we propose to map the rank position to a continuous score variable that represents the qualification of the candidates. Then, we build a causal graph that consists of both the discrete profile attributes and the continuous score. The path-specific effect technique is extended to the mixed-variable causal graph to identify both direct and indirect discrimination. The relationship between the path-specific effects for the ranked data and those for the binary decision is theoretically analyzed. Finally, algorithms for discovering and removing discrimination from a ranked dataset are developed. Experiments using the real dataset show the effectiveness of our approaches.

Motivation & Objective

• Address the limitation of statistical parity in fairness-aware ranking, which fails to capture causal discrimination.
• Recognize that existing causal methods for classification cannot be directly applied to ranked data due to the correlated, permutation-based nature of rankings.
• Develop a method to model ranking positions as continuous scores to enable causal inference in ranked datasets.
• Extend path-specific effect analysis to mixed-variable causal graphs (categorical attributes + continuous scores) for discrimination detection.
• Propose algorithms to discover and reconstruct fair rankings by removing direct and indirect discrimination.

Proposed method

• Map ranking positions to continuous scores using the Bradley-Terry model, transforming discrete permutations into a continuous qualification space.
• Construct a mixed-variable causal graph combining categorical profile attributes (e.g., race, zip code) and the continuous score variable.
• Extend the path-specific effect technique to handle mixed continuous and discrete variables, enabling decomposition of causal effects.
• Define and compute direct and indirect discrimination via path-specific effects along specific causal paths in the graph.
• Theoretical analysis establishes the relationship between path-specific effects in ranked data and those in binary classification settings.
• Develop an algorithm to detect discrimination by quantifying path-specific effects and another to reconstruct a fair ranking by adjusting scores to eliminate discriminatory paths.

Experimental results

Research questions

• RQ1 How can direct and indirect discrimination in ranked data be formally defined and measured using causal inference?
• RQ2 To what extent do statistical parity-based fairness metrics fail to detect causal discrimination in ranking systems?
• RQ3 Can path-specific effects from causal graphs be adapted to mixed-variable models involving continuous scores and discrete attributes?
• RQ4 What is the theoretical relationship between path-specific effects in ranked data and those in binary classification problems?
• RQ5 Can a fair ranking be reconstructed by modifying scores to eliminate direct and indirect discrimination while preserving merit-based ordering?

Key findings

• The proposed method successfully detects both direct and indirect discrimination in ranked data, which statistical parity-based methods may miss.
• Experiments on the German Credit dataset show that statistical parity can produce misleading fairness assessments—e.g., masking explicit bias when group proportions are balanced.
• The causal graph approach correctly identifies that a ranker giving a 2-point bonus to a protected group exhibits indirect discrimination, even when statistical parity is satisfied.
• The path-specific effect analysis reveals that discrimination in rankings arises through both direct (protected attribute → score) and indirect (protected attribute → other attributes → score) pathways.
• The algorithm for fair ranking reconstruction effectively removes discriminatory effects while maintaining the merit-based ordering of candidates.
• Theoretical analysis confirms that path-specific effects in ranked data generalize the binary classification case, enabling principled fairness assessment in ranking systems.