[Paper Review] A Voting-Based System for Ethical Decision Making

Ritesh Noothigattu, Snehalkumar S. Gaikwad+5 more|arXiv (Cornell University)|Sep 20, 2017

Ethics and Social Impacts of AISocial Sciences19 citations

TL;DR

This paper proposes a voting-based system for ethical decision making in autonomous vehicles by learning societal preferences from 1.3 million pairwise comparisons on the Moral Machine platform. It uses computational social choice to aggregate individual preferences into collective decisions via a tractable model of permutation processes, achieving high accuracy in predicting ethical choices even with limited per-voter data.

ABSTRACT

We present a general approach to automating ethical decisions, drawing on machine learning and computational social choice. In a nutshell, we propose to learn a model of societal preferences, and, when faced with a specific ethical dilemma at runtime, efficiently aggregate those preferences to identify a desirable choice. We provide a concrete algorithm that instantiates our approach; some of its crucial steps are informed by a new theory of swap-dominance efficient voting rules. Finally, we implement and evaluate a system for ethical decision making in the autonomous vehicle domain, using preference data collected from 1.3 million people through the Moral Machine website.

Motivation & Objective

• To address the lack of formal ethical principles in AI by aggregating societal preferences as a practical alternative.
• To develop a scalable framework for ethical decision making in autonomous vehicles using real-world human judgments.
• To enable runtime ethical decisions when technical and legal constraints fail, by deriving collective preferences from large-scale human input.
• To demonstrate that collective societal preferences can serve as a reliable approximation of ethical norms in the absence of universally agreed principles.

Proposed method

• Collects pairwise ethical judgments from 1.3 million users via the Moral Machine website, representing diverse societal preferences.
• Models individual preferences using a generalized random utility model (transparency model) that supports continuous and discrete features.
• Summarizes individual preference models into a single collective model using a convex optimization framework for maximum likelihood estimation.
• Applies voting rules to the collective model to aggregate preferences over finite subsets of alternatives at runtime.
• Employs a theory of swap-dominance efficient voting rules to ensure fairness and consistency in aggregation.
• Validates the approach using real-world data and shows high predictive accuracy despite low average comparisons per user (≈14).

Experimental results

Research questions

• RQ1 Can societal preferences be reliably aggregated to inform ethical decisions in autonomous vehicles when no ground-truth ethical principles exist?
• RQ2 How can large-scale human judgments be modeled and summarized efficiently to support real-time ethical decision making?
• RQ3 What voting rules and aggregation techniques ensure fairness and consistency in collective ethical decisions derived from diverse human preferences?
• RQ4 To what extent can a single collective model accurately represent the preferences of millions of individuals with limited pairwise comparisons per user?
• RQ5 How can ethical decision-making systems be designed to remain robust and scalable while incorporating future formal ethical or legal principles?

Key findings

• The system achieved high predictive accuracy in inferring ethical preferences, with models trained on ~14 pairwise comparisons per user showing strong performance due to correlated voter parameter distributions.
• The use of a mixture of K transparency models (K > 1) could potentially improve accuracy over a single model, though at the cost of increased computational complexity.
• Empirical results demonstrate that noise in individual responses largely cancels out during summarization and aggregation, supporting the robustness of the collective model.
• The framework enables fallback to societal consensus when ethical or legal principles are unavailable, ensuring decision-making capability in all scenarios.
• The approach is modular and extensible, allowing for future integration of formal ethical or legal rules without disrupting the core decision-making pipeline.
• The method is computationally efficient and scalable, enabling real-time decision-making using a convex optimization-based learning procedure.