[Paper Review] Evaluating a Methodology for Increasing AI Transparency: A Case Study

David Piorkowski, John R. Richards+1 more|arXiv (Cornell University)|Jan 24, 2022

Artificial Intelligence in Healthcare and Education4 citations

TL;DR

This paper evaluates a user-centered methodology for creating AI FactSheets—tailored documentation templates that address specific stakeholder needs in AI development. The methodology enabled non-experts to collaboratively design reusable, domain-specific FactSheets for healthcare AI models, resulting in high-quality, consumer-aligned documentation that outperformed prior practices.

ABSTRACT

In reaction to growing concerns about the potential harms of artificial intelligence (AI), societies have begun to demand more transparency about how AI models and systems are created and used. To address these concerns, several efforts have proposed documentation templates containing questions to be answered by model developers. These templates provide a useful starting point, but no single template can cover the needs of diverse documentation consumers. It is possible in principle, however, to create a repeatable methodology to generate truly useful documentation. Richards et al. [25] proposed such a methodology for identifying specific documentation needs and creating templates to address those needs. Although this is a promising proposal, it has not been evaluated. This paper presents the first evaluation of this user-centered methodology in practice, reporting on the experiences of a team in the domain of AI for healthcare that adopted it to increase transparency for several AI models. The methodology was found to be usable by developers not trained in user-centered techniques, guiding them to creating a documentation template that addressed the specific needs of their consumers while still being reusable across different models and use cases. Analysis of the benefits and costs of this methodology are reviewed and suggestions for further improvement in both the methodology and supporting tools are summarized.

Motivation & Objective

• Address the growing societal demand for AI transparency by creating documentation that meets diverse stakeholder needs.
• Overcome limitations of one-size-fits-all AI documentation templates that fail to address specific consumer requirements.
• Evaluate a user-centered methodology for generating FactSheets that are both tailored to consumer needs and reusable across models.
• Assess the practical usability and perceived value of the methodology in a real-world AI healthcare development context.

Proposed method

• Apply a human-centered design methodology to identify documentation needs of AI model producers and consumers.
• Establish a FactSheet team (FS team) to act as intermediaries between content producers and consumers.
• Use iterative elicitation techniques to gather and formalize consumer needs into structured FactSheet template fields.
• Collaborate with producers and consumers to refine and complete the FactSheet templates, ensuring relevance and accuracy.
• Design templates to be specific to the healthcare domain while maintaining reusability across different AI models and use cases.
• Integrate feedback loops to improve template completeness, clarity, and alignment with stakeholder expectations.

Experimental results

Research questions

• RQ1 Is the FactSheets methodology usable by FS team members without formal training in human-centered design?
• RQ2 How well do the resulting FactSheets address the specific documentation needs of diverse consumers?
• RQ3 What are the perceived benefits and costs of using the methodology from the perspectives of both producers and consumers?

Key findings

• FS team members without prior training in human-centered design successfully applied the methodology to elicit consumer needs and create effective FactSheet templates.
• The FactSheets produced were perceived as high-quality and significantly improved upon previous documentation practices, with 16 out of 17 participants agreeing they were an improvement.
• The resulting FactSheets addressed domain-specific needs in healthcare AI while remaining general enough for reuse across multiple, dissimilar models.
• Consumers reported that the FactSheets met their needs effectively, although some documentation gaps were identified, reinforcing the need for tailored documentation.
• Participants identified key benefits including centralized, authoritative documentation and support for deeper exploration of model facts, with one participant calling the methodology 'a core part of what we’re doing.'
• The methodology’s benefits—such as improved clarity, traceability, and stakeholder alignment—were consistently reported to outweigh the costs, despite some effort required in initial elicitation and iteration.