[Paper Review] Responsible AI by Design in Practice

Richard Benjamins, Alberto Barbado+1 more|arXiv (Cornell University)|Sep 27, 2019

Ethics and Social Impacts of AISocial Sciences35 references36 citations

TL;DR

The paper presents a practical case of implementing a company-wide methodology to minimize undesired AI consequences in a large organization, highlighting technical and organizational steps.

ABSTRACT

Recently, a lot of attention has been given to undesired consequences of Artificial Intelligence (AI), such as unfair bias leading to discrimination, or the lack of explanations of the results of AI systems. There are several important questions to answer before AI can be deployed at scale in our businesses and societies. Most of these issues are being discussed by experts and the wider communities, and it seems there is broad consensus on where they come from. There is, however, less consensus on, and experience with how to practically deal with those issues in organizations that develop and use AI, both from a technical and organizational perspective. In this paper, we discuss the practical case of a large organization that is putting in place a company-wide methodology to minimize the risk of undesired consequences of AI. We hope that other organizations can learn from this and that our experience contributes to making the best of AI while minimizing its risks.

Motivation & Objective

• Motivate the need to address bias, discrimination, and explanations in AI deployments.
• Describe how a large organization structures and executes a company-wide responsible AI methodology.
• Share lessons learned from applying responsible AI practices in a real-world setting.

Proposed method

• Present a practical case study of a large organization adopting a responsible AI design methodology.
• Discuss the integration of technical and organizational approaches to minimize AI risks.
• Highlight governance structures, processes, and workflows used to operationalize responsible AI.
• Provide insights drawn from experience within the Human-Centered AI: Trustworthiness track at AAAI Fall Symposium.

Experimental results

Research questions

• RQ1 What practical steps can organizations take to minimize undesired consequences of AI in real-world deployments?
• RQ2 How can technical and organizational practices be combined to design trustworthy AI at scale?
• RQ3 What governance, risk management, and measurement approaches are effective for responsible AI in organizations?

Key findings

• A real-world case demonstrates how a large organization can implement a company-wide responsible AI methodology.
• Insights into organizational challenges and enablers for operationalizing responsible AI.
• Discussion of governance, processes, and workflows necessary to minimize AI risks in practice.
• Evidence drawn from participation in the HAi track at the AAAI Fall Symposium on Trustworthy AI Models & Data.