[Paper Review] Conceptualising Contestability: Perspectives on Contesting Algorithmic Decisions

Henrietta Lyons, Eduardo Velloso+1 more|arXiv (Cornell University)|Feb 23, 2021

Ethics and Social Impacts of AISocial Sciences118 references73 citations

TL;DR

This paper investigates how contestability—defined as the ability to challenge algorithmic decisions—is conceptualized by diverse stakeholders in response to Australia’s AI Ethics Framework, the first such framework to enshrine contestability as a core principle. Through thematic analysis of 65 submissions, the study reveals broad agreement that contestability must protect individuals and mirror human decision contestation processes, while highlighting key design and policy challenges for implementing fair, accessible, and explainable contestation mechanisms in AI systems.

ABSTRACT

As the use of algorithmic systems in high-stakes decision-making increases, the ability to contest algorithmic decisions is being recognised as an important safeguard for individuals. Yet, there is little guidance on what `contestability'--the ability to contest decisions--in relation to algorithmic decision-making requires. Recent research presents different conceptualisations of contestability in algorithmic decision-making. We contribute to this growing body of work by describing and analysing the perspectives of people and organisations who made submissions in response to Australia's proposed `AI Ethics Framework', the first framework of its kind to include `contestability' as a core ethical principle. Our findings reveal that while the nature of contestability is disputed, it is seen as a way to protect individuals, and it resembles contestability in relation to human decision-making. We reflect on and discuss the implications of these findings.

Motivation & Objective

• . The research addresses the lack of clear guidance on what 'contestability' means in algorithmic decision-making.
• It investigates how diverse stakeholders—including government, industry, academia, and civil society—conceptualize contestability in response to Australia’s AI Ethics Framework.
• The study aims to map the conceptual debates and practical challenges surrounding contestation in AI systems.
• It seeks to inform the design of fair, accessible, and transparent contestation processes for algorithmic decisions.
• The objective is to contribute to ethical AI development by identifying core principles and design considerations for contestability.

Proposed method

• . The study analyzed 65 submissions to Australia’s AI Ethics Framework consultation process.
• A thematic analysis was conducted to identify recurring perspectives, conceptualizations, and design requirements for contestability.
• The analysis focused on how respondents framed contestability, its relationship to fairness and transparency, and its operationalization.
• The researchers examined the role of explainability in enabling contestation, particularly through human-centered explanations.
• They assessed how contestation processes could be designed to be accessible and effective, drawing on insights from HCI and policy design.
• The findings were used to reflect on the feasibility and implications of contestability as a safeguard in AI systems.

Experimental results

Research questions

• RQ1 . How do diverse stakeholders conceptualize contestability in the context of algorithmic decision-making?
• RQ2 What are the key design and policy considerations for implementing contestation processes in AI systems?
• RQ3 How does contestability in algorithmic decisions compare to contestability in human decision-making?
• RQ4 What role does explainability play in enabling or limiting the grounds for contestation?
• RQ5 What are the systemic challenges that hinder individuals from effectively contesting algorithmic decisions?

Key findings

• . There is broad consensus that contestability is essential for protecting individuals from unfair or harmful algorithmic decisions.
• . Respondents consistently viewed contestability in algorithmic decisions as analogous to contestability in human decision-making, emphasizing similar procedural fairness.
• . The submissions emphasized that contestation processes must be accessible, transparent, and designed with input from affected individuals.
• . A major challenge identified was the opacity of algorithmic systems, which hinders users’ ability to understand or challenge decisions.
• . There is a strong need for human-centered explanations that go beyond technical or counterfactual explanations to support meaningful contestation.
• . The study highlights that current explainable AI (XAI) outputs may be insufficient for legal or formal contestation, suggesting a need for more robust, context-aware explanation frameworks.