[论文解读] A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Jules White, Quchen Fu+7 more|arXiv (Cornell University)|Feb 21, 2023

Software Engineering Research被引用 771

一句话总结

论文呈现一个提示模式目录——组织为可重用解决方案——提升 ChatGPT 的提示工程，包括用于记录模式及其在软件开发任务中的组合框架。

ABSTRACT

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

研究动机与目标

• 提供一个记录提示模式以解决一系列LLM交互问题的框架。
• 引入一套应用于自动化软件开发任务的领域无关提示模式目录。
• 展示如何从多种模式构建提示并与其他模式结合以提升输出。

提出的方法

• 定义一个基于模式的框架，使提示的结构类似于软件设计模式。
• 对应用于对话式LLMs和输出生成的16种提示模式进行分类和编目。
• 描述模式结构、意图、动机及示例实现以实现复用。

实验结果

研究问题

• RQ1 如何对提示模式进行文档化并跨领域迁移？
• RQ2 在软件任务中改进LLM输出和交互的关键提示模式有哪些？
• RQ3 如何将多种提示模式组合以实现更广泛的目标？

主要发现

• 引入一个由16种提示模式组成的目录，分为五组：输入语义、输出定制、错误识别、提示改进和交互。
• 定义一个元语言创建模式，以允许提示的替代语言或记号。
• 提出 Output Automater、Persona、Visualization Generator、Recipe、Template、Fact Check List、Reflection、Question Refinement、Alternative Approaches、Cognitive Verifier、Refusal Breaker、Flipped Interaction、Game Play、Infinite Generation 和 Context Manager 等模式。
• 将基本的上下文陈述描述为一种记录提示想法的轻量级方法，避免刚性语法。
• 表明提示可以由多种模式组成，以满足复杂的交互和输出需求。
• 指出使用 ChatGPT+ 进行评估，并强调模式的复用、可移植性和领域适应性。