[Paper Review] On the Creativity of Large Language Models

Giorgio Franceschelli, Mirco Musolesi|arXiv (Cornell University)|Mar 27, 2023

Machine Learning in Materials Science9 citations

TL;DR

The paper analyzes whether large language models (LLMs) can be creative by applying Boden’s criteria of value, novelty, and surprise, and discusses the requirements for true creativity beyond product alone.

ABSTRACT

Large Language Models (LLMs) are revolutionizing several areas of Artificial Intelligence. One of the most remarkable applications is creative writing, e.g., poetry or storytelling: the generated outputs are often of astonishing quality. However, a natural question arises: can LLMs be really considered creative? In this article, we first analyze the development of LLMs under the lens of creativity theories, investigating the key open questions and challenges. In particular, we focus our discussion on the dimensions of value, novelty, and surprise as proposed by Margaret Boden in her work. Then, we consider different classic perspectives, namely product, process, press, and person. We discuss a set of ``easy'' and ``hard'' problems in machine creativity, presenting them in relation to LLMs. Finally, we examine the societal impact of these technologies with a particular focus on the creative industries, analyzing the opportunities offered, the challenges arising from them, and the potential associated risks, from both legal and ethical points of view.

Motivation & Objective

• Assess LLMs against Boden’s creativity criteria (value, novelty, surprise).
• Examine classic creativity perspectives (product, process, press, person) in the LLM context.
• Identify easy and hard problems in machine creativity relevant to LLMs.
• Discuss practical legal, ethical, and societal impacts on creative industries.

Proposed method

• Review Boden’s three criteria for creativity and related theories (Amabile, Csikszentmihalyi, Gaut).
• Map LLM capabilities to value, novelty, and surprise considering their autoregressive nature.
• Discuss implications of product, process, press, and person perspectives for LLM creativity.
• Differentiate between combinatorial, exploratory, and transformational creativity in LLM outputs.

Experimental results

Research questions

• RQ1 Can LLMs produce artifacts that are new, surprising, and valuable according to Boden’s criteria?
• RQ2 What are the limitations of LLMs in achieving true creativity (especially transformational creativity)?
• RQ3 How do product, process, press, and person perspectives apply to LLM creativity?
• RQ4 What are the societal, legal, and ethical implications of LLM creativity for creative industries?

Key findings

• LLMs can produce valuable outputs but struggle with novelty and surprise under current autoregressive training.
• Transformational creativity appears unlikely with current training paradigms; combinatorial creativity may be possible with prompting or conditioning.
• LLMs lack intrinsic motivation, self-directed process, and self-evaluation required for true creative processes.
• The social and environmental (press) dimension of creativity is difficult for immutable LLMs to participate in meaningfully.
• Continual learning and adaptation are proposed as future directions to foster closer alignment with a never-ending creative cycle.
• Legal and ethical considerations, including copyright and attribution, are central to deploying LLMs in creative work.