[Paper Review] Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systems

Zhiyu Lin, Upol Ehsan+4 more|arXiv (Cornell University)|May 3, 2023

Creativity in Education and Neuroscience11 citations

TL;DR

The paper defines a design space for mixed-initiative co-creativity (MI-CC) systems, implements 7 system variants to cover the space, and shows that broader design-space coverage improves perceived creativity support and achievement, with user preferences varying by expertise.

ABSTRACT

Generative Artificial Intelligence systems have been developed for image, code, story, and game generation with the goal of facilitating human creativity. Recent work on neural generative systems has emphasized one particular means of interacting with AI systems: the user provides a specification, usually in the form of prompts, and the AI system generates the content. However, there are other configurations of human and AI coordination, such as co-creativity (CC) in which both human and AI systems can contribute to content creation, and mixed-initiative (MI) in which both human and AI systems can initiate content changes. In this paper, we define a hypothetical human-AI configuration design space consisting of different means for humans and AI systems to communicate creative intent to each other. We conduct a human participant study with 185 participants to understand how users want to interact with differently configured MI-CC systems. We find out that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics, demonstrating that wider coverage of the design space can improve user experience and achievement when using the system; Preference varies greatly between expertise groups, suggesting the development of adaptive, personalized MI-CC systems; Participants identified new design space dimensions including scrutability -- the ability to poke and prod at models -- and explainability.

Motivation & Objective

• Propose and operationalize a design-space framework for MI-CC systems (three axes: initiator, elaboration/reflection, global/local).
• Investigate how different communication configurations affect user-perceived creativity support and goal attainment in story generation.
• Assess whether broader design-space coverage improves user experience across diverse user expertise levels.
• Identify new design-space dimensions reported by users (e.g., explainability/scrutability) and their impact on collaboration.

Proposed method

• Construct a hypothetical MI-CC design space with three axes (Human vs. Agent-initiated, Elaboration vs. Reflection, Global vs. Local).
• Instantiate 7 MI-CC storytelling system variants representing subsets of the design space for an exploratory study.
• Leverage the Creative Wand framework with four components (Creative Context, Experience Manager, Communications, Frontend) to run experiments.
• Use two AI storytelling systems (Plug and Blend with topic control and GPT-J, and CARP for critique) to implement communication modules.
• Measure outcomes with the Creative Support Index and task performance on a constrained 10-line story.
• Conduct a between-subjects study with 185 participants recruited via Prolific, randomizing system condition and counter-balancing presentation order.

Experimental results

Research questions

• RQ1 Does broader coverage of the MI-CC design space improve perceived creative support and goal achievement?
• RQ2 How do different communication types (agent-initiated, human-initiated, elaboration, reflection, global, local) influence user experience across expertise levels?
• RQ3 What new dimensions (e.g., scrutability, explainability) emerge from user studies of MI-CC systems?

Key findings

• Wider design-space coverage yields equal or better ratings on creative support and perceived achievement across multiple metrics.
• Removing dimensions such as agent-initiated or elaboration communications degrades the creative experience across measures of expressiveness, enjoyment, exploration, immersion, collaboration, and perceived result quality.
• Global-only communications reduce exploration and perceived results worth the effort, highlighting the importance of local changes for creativity exploration.
• Participant preferences vary by expertise and familiarity with AI, suggesting the need for adaptive, personalized MI-CC configurations.
• Users value controllability and scrutability, and express a desire for explainability to build trust and improve mental models of AI behavior.