[Paper Review] Fairness in Learning-Based Sequential Decision Algorithms: A Survey

Xueru Zhang, Mingyan Liu|arXiv (Cornell University)|Jan 14, 2020

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This survey examines fairness in learning-based sequential decision-making, distinguishing between settings where past decisions do not affect future data and those where they do. It shows that fairness interventions—especially group fairness constraints like demographic parity—can exacerbate representation disparity and worsen feature distribution skew over time, particularly when decisions reshape individual behavior.

ABSTRACT

Algorithmic fairness in decision-making has been studied extensively in static settings where one-shot decisions are made on tasks such as classification. However, in practice most decision-making processes are of a sequential nature, where decisions made in the past may have an impact on future data. This is particularly the case when decisions affect the individuals or users generating the data used for future decisions. In this survey, we review existing literature on the fairness of data-driven sequential decision-making. We will focus on two types of sequential decisions: (1) past decisions have no impact on the underlying user population and thus no impact on future data; (2) past decisions have an impact on the underlying user population and therefore the future data, which can then impact future decisions. In each case the impact of various fairness interventions on the underlying population is examined.

Motivation & Objective

• To analyze the long-term effects of fairness interventions in sequential decision-making systems where decisions influence future data.
• To distinguish between two settings: (1) decisions do not affect future data, and (2) decisions alter the underlying population and thus future data distributions.
• To investigate how fairness constraints (e.g., demographic parity, equalized odds) affect group representation and feature evolution over time.
• To highlight the risk of fairness interventions reinforcing or worsening bias due to feedback loops between decisions and population behavior.
• To provide a structured review of reinforcement learning and MDP-based frameworks for modeling fairness in dynamic, feedback-driven environments.

Proposed method

• Models sequential decision-making using Markov Decision Processes (MDPs), where states include individual features and sensitive attributes, and actions represent decisions.
• Introduces two scenarios for how decisions affect feature distributions: (1) group proportions change due to perceived loss, (2) individuals from disadvantaged groups adapt behavior to reduce loss, altering feature distributions.
• Analyzes fairness constraints such as demographic parity (DP) and equalized odds (EqOpt) under time-varying feature distributions.
• Uses mathematical modeling to show that fairness interventions can accelerate representation disparity when feature distributions evolve due to decision feedback.
• Applies reinforcement learning frameworks from jabbari2017fairness and wen2019fairness, incorporating long-term reward fairness and group fairness constraints.
• Evaluates the impact of fairness on learning convergence, showing that stricter fairness requirements increase the time to achieve near-optimality.

Experimental results

Research questions

• RQ1 How do fairness interventions in sequential decision-making affect long-term group representation disparity when decisions influence future data?
• RQ2 What happens to feature distributions in disadvantaged groups when individuals adapt their behavior in response to repeated unfair decisions?
• RQ3 Can fairness constraints like demographic parity or equalized odds lead to unintended worsening of bias over time in feedback-driven systems?
• RQ4 How does the interplay between decision-making and population behavior create feedback loops that reinforce or amplify existing disparities?
• RQ5 In what ways do fairness-aware reinforcement learning algorithms differ in performance and fairness outcomes when feature distributions evolve over time?

Key findings

• Fairness interventions such as demographic parity (DP) and equalized odds (EqOpt) can worsen representation disparity over time when decisions affect the underlying population.
• When individuals from disadvantaged groups adapt their behavior to reduce decision-related losses, their feature distributions skew in ways that can further disadvantage them, accelerating disparity.
• The combination of feedback from decisions and evolving feature distributions leads to a feedback loop that amplifies bias, especially under group fairness constraints.
• Even with fairness constraints, the time to learn a near-optimal decision rule increases significantly when fairness requirements are stricter, as shown in reinforcement learning frameworks.
• In settings where decisions alter group proportions or feature distributions, fairness interventions that assume static distributions may fail to prevent long-term inequity.
• The study demonstrates that fairness in sequential systems cannot be ensured by static fairness definitions alone—dynamic modeling of population feedback is essential.