[论文解读] Automating Creativity

Minghui Huang, Roland T. Rust|arXiv (Cornell University)|May 11, 2024

Creativity in Education and Neuroscience被引用 6

一句话总结

论文提出一个三重提示-回应-奖励工程框架，通过强化学习和计算创造力的洞见，将生成式AI从单纯生成提升到创造力。

ABSTRACT

Generative AI (GenAI) has spurred the expectation of being creative, due to its ability to generate content, yet so far, its creativity has somewhat disappointed, because it is trained using existing data following human intentions to generate outputs. The purpose of this paper is to explore what is required to evolve AI from generative to creative. Based on a reinforcement learning approach and building upon various research streams of computational creativity, we develop a triple prompt-response-reward engineering framework to develop the creative capability of GenAI. This framework consists of three components: 1) a prompt model for expected creativity by developing discriminative prompts that are objectively, individually, or socially novel, 2) a response model for observed creativity by generating surprising outputs that are incrementally, disruptively, or radically innovative, and 3) a reward model for improving creativity over time by incorporating feedback from the AI, the creator/manager, and/or the customers. This framework enables the application of GenAI for various levels of creativity strategically.

研究动机与目标

• 动机：推动将 GenAI 从生成内容转向展现创造力。
• 提出一个框架，利用提示、回应和奖励来培养创造性 AI。
• 整合来自 AI、创作者/管理者和客户的反馈，以随着时间的推移提升创造力。

提出的方法

• 开发一个提示模型，能够为目标、个人或社会新颖性生成具有辨别性的提示。
• 开发一个回应模型，生成的输出具有惊讶性并逐步创新（渐进、颠覆、根本性）。
• 开发一个奖励模型，整合多源反馈以随时间提高创造力。

实验结果

研究问题

• RQ1 如何通过提示设计将 GenAI 引导至不同层次的创造力（渐进到根本性）？
• RQ2 哪些机制通过反馈回路使观察到的创造力随时间提高？
• RQ3 如何在 GenAI 创造力的强化学习方法中实例化三重提示-回应-奖励框架？

主要发现

• 提出三重提示-回应-奖励框架，以在 GenAI 中实现创造性能力。
• 该框架区分用于新颖性的提示、具有惊讶性的回应，以及随时间提升创造力的奖励。
• 它使得有策略地将 GenAI 应用于不同水平的创造力成为可能。