[Paper Review] Wordflow: Social Prompt Engineering for Large Language Models

Zijie J. Wang, Aishwarya Chakravarthy+2 more|arXiv (Cornell University)|Jan 25, 2024

Topic Modeling4 citations

TL;DR

This paper introduces Wordflow, a social prompt engineering platform that enables non-expert users to collaboratively create, run, share, and discover large language model (LLM) prompts through an open-source, browser-based text editor. By integrating local LLM inference via WebGPU and social computing features like prompt sharing and tagging, Wordflow lowers barriers to effective prompting and supports everyday users in applying LLMs to real-world tasks such as proofreading and creative writing.

ABSTRACT

Large language models (LLMs) require well-crafted prompts for effective use. Prompt engineering, the process of designing prompts, is challenging, particularly for non-experts who are less familiar with AI technologies. While researchers have proposed techniques and tools to assist LLM users in prompt design, these works primarily target AI application developers rather than non-experts. To address this research gap, we propose social prompt engineering, a novel paradigm that leverages social computing techniques to facilitate collaborative prompt design. To investigate social prompt engineering, we introduce Wordflow, an open-source and social text editor that enables everyday users to easily create, run, share, and discover LLM prompts. Additionally, by leveraging modern web technologies, Wordflow allows users to run LLMs locally and privately in their browsers. Two usage scenarios highlight how social prompt engineering and our tool can enhance laypeople's interaction with LLMs. Wordflow is publicly accessible at https://poloclub.github.io/wordflow.

Motivation & Objective

• To address the challenge of prompt engineering for non-expert users who lack training and guidance in using large language models (LLMs).
• To bridge the research gap by focusing on everyday users rather than AI developers, who are the primary focus of existing prompt engineering tools.
• To introduce and evaluate social prompt engineering—a novel paradigm that leverages collaborative, community-driven techniques to improve prompt design and usage.
• To develop Wordflow, an open-source, web-based text editor that enables local, private LLM inference and supports social interactions around prompt creation and sharing.
• To investigate how social computing features enhance user engagement, prompt quality, and accessibility in LLM interaction.

Proposed method

• Wordflow is a browser-based, open-source text editor that allows users to compose, edit, and execute LLM prompts directly on their devices using WebGPU for local inference.
• The platform supports direct manipulation of text: users highlight text and apply prompts to transform content (e.g., for proofreading or translation) with one-click execution.
• Users can customize prompts and LLM settings, save them to the Prompt Manager, and share them with the community using tags and metadata.
• The tool enables users to discover and reuse community-shared prompts, with sorting and filtering by popularity, recency, and tags.
• Local LLM execution ensures data privacy and eliminates dependency on external APIs or backend servers.
• Social features include prompt sharing, community curation, and user reporting of harmful content, with plans for content moderation and reputation systems.

Experimental results

Research questions

• RQ1 How does social prompt engineering support non-expert users in crafting effective LLM prompts?
• RQ2 What are the common use cases and prompting patterns employed by everyday users when interacting with LLMs?
• RQ3 How do users modify and re-share community prompts, and what does this reveal about collaborative learning in prompting?
• RQ4 What role do social features like tagging, popularity ranking, and reputation systems play in enhancing user engagement and prompt discovery?
• RQ5 How can social systems be designed to promote responsible prompting and mitigate risks such as misinformation or harmful outputs?

Key findings

• Wordflow enables non-expert users to create, run, and share LLM prompts locally in the browser using WebGPU, eliminating reliance on external APIs and enhancing data privacy.
• The platform’s social features—such as prompt sharing, tagging, and community curation—facilitate collaborative learning and discovery of effective prompting techniques.
• Users can filter and sort prompts by popularity, recency, and tags, enabling efficient discovery of high-quality, contextually relevant prompts.
• The tool supports direct manipulation of text with one-click LLM execution, significantly lowering the barrier to entry for non-technical users.
• Preliminary usage log studies are planned to analyze prompt evolution, including how users adapt and re-share community prompts.
• The platform includes mechanisms for reporting harmful prompts, signaling a foundation for responsible AI practices in social prompting systems.