[Paper Review] Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities

Samer B. Nashed, Justin Svegliato+1 more|arXiv (Cornell University)|Jan 13, 2023

Ethics and Social Impacts of AI4 citations

TL;DR

This paper bridges algorithmic fairness and ethical decision-making by analyzing how fairness concepts from predictive systems can inform sequential decision-making models—such as Markov Decision Processes—and how ethical decision-making techniques can enhance fairness research. It identifies key limitations in applying fairness methods to sequential systems, highlights opportunities for cross-fertilization, and calls for improved interpretability, stakeholder engagement, and ethical analysis in long-term decision systems.

ABSTRACT

As automated decision making and decision assistance systems become common in everyday life, research on the prevention or mitigation of potential harms that arise from decisions made by these systems has proliferated. However, various research communities have independently conceptualized these harms, envisioned potential applications, and proposed interventions. The result is a somewhat fractured landscape of literature focused generally on ensuring decision-making algorithms "do the right thing". In this paper, we compare and discuss work across two major subsets of this literature: algorithmic fairness, which focuses primarily on predictive systems, and ethical decision making, which focuses primarily on sequential decision making and planning. We explore how each of these settings has articulated its normative concerns, the viability of different techniques for these different settings, and how ideas from each setting may have utility for the other.

Motivation & Objective

• To examine the conceptual, methodological, and practical gaps between algorithmic fairness and ethical decision-making research communities.
• To assess the viability of fairness techniques—such as bias mitigation and fairness metrics—within sequential decision-making frameworks like Markov Decision Processes (MDPs).
• To identify how ethical decision-making approaches, including value alignment and long-term consequence modeling, can inform fairness research in dynamic systems.
• To highlight underexplored opportunities in interpretability, human-in-the-loop design, and stakeholder engagement for sequential decision-making systems.
• To advocate for integrated, interdisciplinary approaches to ensure ethical, transparent, and accountable deployment of sequential AI systems in high-stakes domains.

Proposed method

• Uses the Markov Decision Process (MDP) as a foundational model for sequential decision making, analyzing its structure, solution methods, and assumptions.
• Compares normative concerns, fairness metrics, and mitigation strategies across algorithmic fairness and ethical decision-making literatures.
• Analyzes the sequential decision-making pipeline, including model design, reward shaping, and policy learning, to identify points of ethical risk and intervention.
• Evaluates existing fairness metrics (e.g., demographic parity, equal opportunity) for applicability in sequential settings, noting limitations due to temporal and dynamic dependencies.
• Introduces the role of interpretability and explainability in sequential systems, drawing on concepts like actionable recourse and cross-examination from fairness literature.
• Proposes integrating participatory design and stakeholder engagement practices from human-computer interaction into AI system development for ethical alignment.

Experimental results

Research questions

• RQ1 To what extent can fairness metrics and mitigation techniques from predictive systems be meaningfully applied to sequential decision-making models?
• RQ2 How do ethical decision-making frameworks, such as value alignment and long-term consequence modeling, inform fairness in dynamic, multi-step systems?
• RQ3 What are the key differences in normative concerns and design challenges between fairness in predictive systems and ethical decision-making in sequential systems?
• RQ4 How can interpretability and human-in-the-loop mechanisms be adapted for sequential decision-making systems to improve accountability and transparency?
• RQ5 What role can stakeholder engagement and participatory design play in shaping ethical and fair sequential decision-making systems?

Key findings

• Fairness metrics such as demographic parity and equal opportunity are often ill-suited for sequential decision-making systems due to their static, one-shot nature and failure to account for long-term dynamics.
• Ethical decision-making techniques, including reward shaping and value alignment, offer promising tools for modeling fairness over time, particularly in systems with feedback loops and cumulative impacts.
• Interpretability and explainability in sequential systems remain underdeveloped compared to predictive models, with limited research on actionable recourse or cross-examination in dynamic settings.
• Current practices in model development often exclude stakeholders, despite the potential for expert and community knowledge to improve ethical outcomes in sequential systems.
• The lack of standardized ethical analysis in sequential systems—especially regarding long-term harm and systemic injustice—poses significant risks, particularly in domains like autonomous driving and healthcare.
• The integration of fairness and ethical decision-making practices is hindered by differing assumptions, design pipelines, and deployment contexts, requiring new interdisciplinary frameworks to bridge the gap.