[论文解读] Unleashing the potential of prompt engineering for large language models

B.‐C. CHEN, Zhaofeng Zhang+2 more|arXiv (Cornell University)|Oct 23, 2023

Topic Modeling被引用 133

一句话总结

对大型语言模型的提示工程技术的综合综述，涵盖基础方法、进阶提示策略、评估和应用。

ABSTRACT

This comprehensive review delves into the pivotal role of prompt engineering in unleashing the capabilities of Large Language Models (LLMs). The development of Artificial Intelligence (AI), from its inception in the 1950s to the emergence of advanced neural networks and deep learning architectures, has made a breakthrough in LLMs, with models such as GPT-4o and Claude-3, and in Vision-Language Models (VLMs), with models such as CLIP and ALIGN. Prompt engineering is the process of structuring inputs, which has emerged as a crucial technique to maximize the utility and accuracy of these models. This paper explores both foundational and advanced methodologies of prompt engineering, including techniques such as self-consistency, chain-of-thought, and generated knowledge, which significantly enhance model performance. Additionally, it examines the prompt method of VLMs through innovative approaches such as Context Optimization (CoOp), Conditional Context Optimization (CoCoOp), and Multimodal Prompt Learning (MaPLe). Critical to this discussion is the aspect of AI security, particularly adversarial attacks that exploit vulnerabilities in prompt engineering. Strategies to mitigate these risks and enhance model robustness are thoroughly reviewed. The evaluation of prompt methods is also addressed through both subjective and objective metrics, ensuring a robust analysis of their efficacy. This review also reflects the essential role of prompt engineering in advancing AI capabilities, providing a structured framework for future research and application.

研究动机与目标

• 解释面向大型语言模型的提示工程的基础原理与组成要素。
• 调研基本和高级提示技术，以提高输出质量并减少幻觉。
• 讨论外部辅助工具，如插件和检索增强等，以提升提示效果。
• 识别从多角度评估提示有效性的方法。
• 探索提示工程在教育、编程等领域的应用。

提出的方法

• 描述基本的提示要素，以及指令质量如何影响输出。
• 介绍角色提示、单-shot 和少-shot 提示，以及提示结构的影响。
• 解释高级技巧：思维链、自一致性、生成知识、由少到多提示、思维树、思维图。
• 讨论检索增强与插件辅助的提示润色。
• 强调评估策略与未来方向。

Figure 10: Guideline of courses generated by GPT-4

实验结果

研究问题

• RQ1 面向LLM的提示工程的核心原则和基本技巧是什么？
• RQ2 高级提示方法（Chain-of-Thought、ToT、GoT 等）如何提高推理和准确性？
• RQ3 哪些外部辅助工具（检索、插件）可以提升提示有效性并减少幻觉？
• RQ4 如何在不同任务和视角下评估提示有效性？
• RQ5 在各领域中，提示工程的潜在未来发展方向和应用有哪些？

主要发现

• 提示工程涵盖从基本提示构造到以高级推理驱动的技术的广泛范围。
• 如思维链和自一致性等高级方法在各类任务中提升推理准确性。
• 如检索增强和插件等外部辅助可以降低幻觉并提升输出。
• 提示有效性可从多角度评估，包括跨领域的准确性、连贯性和有用性。
• 提示工程在教育、编程及其他领域具有变革性潜力。