[Paper Review] Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure

Ben Hutchinson, Andrew Smart+6 more|arXiv (Cornell University)|Oct 23, 2020

Ethics and Social Impacts of AISocial Sciences129 references46 citations

TL;DR

The paper argues that ML datasets are infrastructural artifacts and proposes a rigorous, lifecycle-based documentation framework inspired by software engineering to ensure transparency, accountability, and responsible dataset development.

ABSTRACT

Rising concern for the societal implications of artificial intelligence systems has inspired demands for greater transparency and accountability. However the datasets which empower machine learning are often used, shared and re-used with little visibility into the processes of deliberation which led to their creation. Which stakeholder groups had their perspectives included when the dataset was conceived? Which domain experts were consulted regarding how to model subgroups and other phenomena? How were questions of representational biases measured and addressed? Who labeled the data? In this paper, we introduce a rigorous framework for dataset development transparency which supports decision-making and accountability. The framework uses the cyclical, infrastructural and engineering nature of dataset development to draw on best practices from the software development lifecycle. Each stage of the data development lifecycle yields a set of documents that facilitate improved communication and decision-making, as well as drawing attention the value and necessity of careful data work. The proposed framework is intended to contribute to closing the accountability gap in artificial intelligence systems, by making visible the often overlooked work that goes into dataset creation.

Motivation & Objective

• Argue that ML datasets function as technical infrastructure requiring visibility and accountability.
• Advocate adopting software engineering lifecycle practices to dataset development.
• Propose a structured documentation model with specific artifact types to enable audits and reviews.
• Highlight the political and engineering dimensions of dataset work and its non-linear lifecycle.

Proposed method

• Framing datasets as infrastructure and engineering artifacts to justify the need for accountability.
• Mapping dataset development stages to a software-like lifecycle (requirements, design, implementation, testing, maintenance).
• Introducing document types at each stage to facilitate traceability and accountability (Requirements Analysis Documents, Dataset Design Documents, Implementation Diaries, Testing Reports, Maintenance Plans).
• Proposing governance concepts such as audits, diverse oversight, and postmortems to address accountability gaps.

Experimental results

Research questions

• RQ1 What information should be recorded to enable meaningful accountability in dataset development ex ante, during, and ex post?
• RQ2 How can software engineering practices be adapted to improve visibility, ownership, and auditability of ML datasets?
• RQ3 What are the key documentation artifacts and ownership roles required across the dataset development lifecycle?
• RQ4 How do the concepts of datasets as infrastructure influence accountability and governance in ML?

Key findings

• Datasets are best viewed as infrastructure that enables ML systems and thus require deliberate, non-hasty development and documentation.
• A non-linear, iterative dataset development lifecycle with explicit ownership and documentation reduces accountability gaps.
• A structured set of documents at each lifecycle stage (requirements, design, implementation, testing, maintenance) supports traceability and accountability.
• Audits, reviews, and ongoing maintenance plans are essential to address data staleness, errors, and changing contexts.
• Documentation should explicitly reflect assumptions, tradeoffs, and stakeholder deliberations to counter biases and unintended harms.