[Paper Review] Automating Creativity

Minghui Huang, Roland T. Rust|arXiv (Cornell University)|May 11, 2024

Creativity in Education and Neuroscience6 citations

TL;DR

The paper proposes a triple prompt-response-reward engineering framework to evolve generative AI from mere generation to creativity, using reinforcement learning and insights from computational creativity.

ABSTRACT

Generative AI (GenAI) has spurred the expectation of being creative, due to its ability to generate content, yet so far, its creativity has somewhat disappointed, because it is trained using existing data following human intentions to generate outputs. The purpose of this paper is to explore what is required to evolve AI from generative to creative. Based on a reinforcement learning approach and building upon various research streams of computational creativity, we develop a triple prompt-response-reward engineering framework to develop the creative capability of GenAI. This framework consists of three components: 1) a prompt model for expected creativity by developing discriminative prompts that are objectively, individually, or socially novel, 2) a response model for observed creativity by generating surprising outputs that are incrementally, disruptively, or radically innovative, and 3) a reward model for improving creativity over time by incorporating feedback from the AI, the creator/manager, and/or the customers. This framework enables the application of GenAI for various levels of creativity strategically.

Motivation & Objective

• Motivate the need to move GenAI from generating content to exhibiting creativity.
• Propose a framework that leverages prompts, responses, and rewards to cultivate creative AI.
• Integrate feedback from AI, creators/managers, and customers to improve creativity over time.

Proposed method

• Develop a prompt model that yields discriminative prompts for objective, individual, or social novelty.
• Develop a response model that generates outputs that are surprising and progressively innovative (incremental, disruptive, radical).
• Develop a reward model that incorporates multi-source feedback to improve creativity over time.

Experimental results

Research questions

• RQ1 How can GenAI be steered to achieve different levels of creativity (incremental to radical) through prompt design?
• RQ2 What mechanisms enable the observed creativity to improve over time via feedback loops?
• RQ3 How can a triple prompt-response-reward framework be instantiated within a reinforcement learning approach for GenAI creativity?

Key findings

• A triple prompt-response-reward framework is proposed to enable creative capabilities in GenAI.
• The framework distinguishes prompts for novelty, responses that are surprising, and rewards that improve creativity over time.
• It enables applying GenAI for various levels of creativity strategically.