[Paper Review] A Word is Worth a Thousand Pictures: Prompts as AI Design Material

Chinmay Kulkarni, Stefania Druga+4 more|arXiv (Cornell University)|Mar 22, 2023

Design Education and Practice18 citations

TL;DR

The paper shows that prompt-based text-to-image models enable non-professional designers to rapidly explore design space and support collaborative design, with prompts acting as reflective design material.

ABSTRACT

Recent advances in Machine-Learning have led to the development of models that generate images based on a text description.Such large prompt-based text to image models (TTIs), trained on a considerable amount of data, allow the creation of high-quality images by users with no graphics or design training. This paper examines the role such TTI models can playin collaborative, goal-oriented design. Through a within-subjects study with 14 non-professional designers, we find that such models can help participants explore a design space rapidly and allow for fluid collaboration. We also find that text inputs to such models ("prompts") act as reflective design material, facilitating exploration, iteration, and reflection in pair design. This work contributes to the future of collaborative design supported by generative AI by providing an account of how text-to-image models influence the design process and the social dynamics around design and suggesting implications for tool design

Motivation & Objective

• Investigate how prompt-based image generation changes the design process for non-professional designers.
• Examine how such models affect collaboration and social dynamics in pair design.
• Characterize prompts as reflective design material that guides exploration and iteration.

Proposed method

• Within-subjects design with 14 non-professional designer pairs.
• Two design sessions per pair using Image Search alone and Image Search plus a prompt-based model (Envisage).
• Participants worked in Google Slides to create invitations, with prompts and images saved for analysis.
• Qualitative analysis of transcripts and interactions to identify emergent themes.
• Post-study surveys and expert ratings to assess creativity, completeness, and appropriateness.

Figure 1 . Large prompt-based text-to-image models enable rapid exploration of a design space, and fluid collaboration. By allowing users to declaratively and quickly create images through text descriptions, these text prompts act as a reflective design material aiding exploration and collaboration.

Experimental results

Research questions

• RQ1 RQ1: How does using prompt-based image generation change the design process of non-professional designers compared to using image search?
• RQ2 RQ2: How do prompt-based image generation models influence collaborative dynamics during design?

Key findings

• Prompts enable rapid, declarative image creation that broadens design exploration.
• Participants perceived higher creativity with Envisage than with image search (average self-rated creativity 3.6 vs 3.0).
• External expert raters did not show significant differences in creativity, completeness, or appropriateness between conditions.
• Design outputs tended to include fewer images in Envisage designs (average 2.4) than in Image Search designs (average 4).
• Prompts function as reflective design material that participants iteratively refined to steer results and understand model behavior.
• Collaboration benefited from shared prompts, but non-determinism and asymmetric access to the model could hinder coordination.