[Paper Review] Hard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical Commitments

Roel Dobbe, Thomas Krendl Gilbert+1 more|arXiv (Cornell University)|Nov 20, 2019

Ethics and Social Impacts of AI3 references4 citations

TL;DR

This paper proposes a framework of sociotechnical commitments to resolve normative uncertainty in AI safety by integrating democratic dissent into AI development. It applies Ruth Chang’s theory of intuitive comparability to identify value dilemmas in design, training, and deployment, and outlines formal, substantive, and discursive commitments to ensure stakeholder accountability and democratic deliberation in AI systems.

ABSTRACT

As AI systems become prevalent in high stakes domains such as surveillance and healthcare, researchers now examine how to design and implement them in a safe manner. However, the potential harms caused by systems to stakeholders in complex social contexts and how to address these remains unclear. In this paper, we explain the inherent normative uncertainty in debates about the safety of AI systems. We then address this as a problem of vagueness by examining its place in the design, training, and deployment stages of AI system development. We adopt Ruth Chang's theory of intuitive comparability to illustrate the dilemmas that manifest at each stage. We then discuss how stakeholders can navigate these dilemmas by incorporating distinct forms of dissent into the development pipeline, drawing on Elizabeth Anderson's work on the epistemic powers of democratic institutions. We outline a framework of sociotechnical commitments to formal, substantive and discursive challenges that address normative uncertainty across stakeholders, and propose the cultivation of related virtues by those responsible for development.

Motivation & Objective

• To address normative uncertainty in AI safety, particularly in high-stakes domains like surveillance and healthcare.
• To identify how vague concepts like 'safety,' 'robustness,' and 'resilience' lead to divergent stakeholder interpretations.
• To propose a structured framework for embedding democratic deliberation and dissent into AI system development.
• To formalize commitments that ensure users can exercise both 'voice' and 'exit' as citizens and consumers.
• To advocate for the integration of sociotechnical commitments into AI education and professional practice.

Proposed method

• Applies Ruth Chang’s theory of intuitive comparability to analyze normative dilemmas in AI design, training, and deployment.
• Classifies normative uncertainty as a form of vagueness, drawing on philosophical models: epistemicism, semantic indeterminism, and incomparability.
• Introduces three types of sociotechnical commitments: formal (institutional structures), substantive (value alignment), and discursive (deliberative channels).
• Proposes mechanisms for user feedback and dissent, including public justification of data use and transparent channels for voicing concerns.
• Models deployment contexts as either market-based (consumer logic) or political (citizen rights), requiring explicit commitment to one or the other.
• Stresses the need for public accountability and trust by preserving both user exit rights and voice mechanisms in system design.

Experimental results

Research questions

• RQ1 How does normative uncertainty manifest in the design, training, and deployment of AI systems across diverse stakeholder perspectives?
• RQ2 What philosophical frameworks can account for the vagueness of core safety concepts like protection, robustness, and resilience?
• RQ3 How can democratic institutions and epistemic virtues be leveraged to resolve value conflicts in AI development?
• RQ4 In what ways do user agreements and data policies reflect either consumer or citizen logics, and what are the implications for accountability?
• RQ5 How can sociotechnical commitments ensure that unrepresented subpopulations are not burdened by unexpressed moral commitments in system specifications?

Key findings

• Normative uncertainty in AI safety arises not from technical flaws but from the inherent vagueness of values like safety, fairness, and resilience across social contexts.
• The concept of intuitive comparability explains why stakeholders often face incommensurable value trade-offs that cannot be resolved by formal metrics alone.
• Sociotechnical commitments—formal, substantive, and discursive—must be institutionalized to embed democratic deliberation into AI development pipelines.
• Preserving both 'voice' and 'exit' is essential for maintaining public accountability, especially when AI systems are deployed as public services or surveillance tools.
• The Rekognition case illustrates how user agreements can be interpreted as private contracts or public assurances, leading to divergent governance models.
• The paper concludes that democratic dissent is not a procedural add-on but a necessary condition for ensuring that AI systems remain aligned with evolving, pluralistic values.