[論文レビュー] A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Jules White, Quchen Fu+7 more|arXiv (Cornell University)|Feb 21, 2023

Software Engineering Research被引用数 771

ひとこと要約

この論文は、再利用可能な解決策として整理されたプロンプトパターンのカタログを提示し、ソフトウェア開発における多様なタスクの組み合わせを可能にするパターンの文書化フレームワークを含む、ChatGPTのプロンプトエンジニアリングを改善します。

ABSTRACT

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

研究の動機と目的

• Provide a framework for documenting prompt patterns to solve a range of LLM interaction problems.
• Introduce a catalog of domain-independent prompt patterns applied to automating software development tasks.
• Show how prompts can be built from multiple patterns and combined with other patterns to enhance outputs.

提案手法

• Define a pattern-based framework that structures prompts similar to software design patterns.
• Classify and catalog 16 prompt patterns applicable to conversational LLMs and output generation.
• Describe pattern structure, intent, motivation, and sample implementations to enable reuse.

実験結果

リサーチクエスチョン

• RQ1 How can prompt patterns be documented and transferred across domains?
• RQ2 What are essential prompt patterns for improving LLM output and interaction in software tasks?
• RQ3 How can multiple prompt patterns be composed to achieve broader goals?

主な発見

• Introduces a catalog of 16 prompt patterns categorized into five groups: Input Semantics, Output Customization, Error Identification, Prompt Improvement, and Interaction.
• Defines a Meta Language Creation pattern to allow alternative languages or notations for prompts.
• Presents Output Automater, Persona, Visualization Generator, Recipe, Template, Fact Check List, Reflection, Question Refinement, Alternative Approaches, Cognitive Verifier, Refusal Breaker, Flipped Interaction, Game Play, Infinite Generation, and Context Manager patterns.
• Describes fundamental contextual statements as a lightweight approach to documenting prompt ideas without rigid grammars.
• Shows that prompts can be composed from multiple patterns to meet complex interaction and output requirements.
• Notes evaluation using ChatGPT+ and emphasizes reuse, transferability, and domain adaptation of the patterns.