[Paper Review] Can AI Be as Creative as Humans?

Haonan Wang, James Zou+9 more|arXiv (Cornell University)|Jan 3, 2024

Machine Learning in Materials Science5 citations

TL;DR

The paper argues AI can reach human-like creativity if it can sufficiently fit conditional data generated by human creators, introducing Relative Creativity and Statistical Creativity with theoretical results and practical guidance for prompting and training autoregressive models.

ABSTRACT

Creativity serves as a cornerstone for societal progress and innovation. With the rise of advanced generative AI models capable of tasks once reserved for human creativity, the study of AI's creative potential becomes imperative for its responsible development and application. In this paper, we prove in theory that AI can be as creative as humans under the condition that it can properly fit the data generated by human creators. Therefore, the debate on AI's creativity is reduced into the question of its ability to fit a sufficient amount of data. To arrive at this conclusion, this paper first addresses the complexities in defining creativity by introducing a new concept called Relative Creativity. Rather than attempting to define creativity universally, we shift the focus to whether AI can match the creative abilities of a hypothetical human. The methodological shift leads to a statistically quantifiable assessment of AI's creativity, term Statistical Creativity. This concept, statistically comparing the creative abilities of AI with those of specific human groups, facilitates theoretical exploration of AI's creative potential. Our analysis reveals that by fitting extensive conditional data without marginalizing out the generative conditions, AI can emerge as a hypothetical new creator. The creator possesses the same creative abilities on par with the human creators it was trained on. Building on theoretical findings, we discuss the application in prompt-conditioned autoregressive models, providing a practical means for evaluating creative abilities of generative AI models, such as Large Language Models (LLMs). Additionally, this study provides an actionable training guideline, bridging the theoretical quantification of creativity with practical model training.

Motivation & Objective

• Propose a framework to evaluate AI creativity relative to human creators using a hypothetical anchor population.
• Define Relative Creativity and Statistical Creativity to quantify AI’s ability to imitate or match human creativity.
• Analyze training data requirements and conditions that enable AI to exhibit creativity comparable to humans.
• Develop theoretical results for autoregressive and prompt-contextualized models applicable to LLMs.
• Offer practical training guidelines emphasizing conditional data and generative processes.

Proposed method

• Introduce Relative Creativity as AI outputs indistinguishable from a plausible human creator given biographical information.
• Define Statistical Creativity via expected evaluator loss over a sample of real human creators and prove a concrete bound (Theorem 1).
• Specialize Statistical Creativity to autoregressive models with a log-likelihood based metric E1(q) and a KL-divergence-based Assumption 1 (Theorem 2).
• Extend the framework to prompt-contextualized settings for LLMs, defining E2, E3 and Corollaries 1 and 2 (Corollaries 1–2).
• Provide a training guideline highlighting the importance of capturing generative conditions and processes, not just raw creation data.
• Discuss connections to generalization and provide Corollary 3 for generalization bounds under a conditional creativity loss.

Experimental results

Research questions

• RQ1 Can an AI model achieve creativity comparable to a hypothetical human creator under plausible evaluation?
• RQ2 What data and training conditions are necessary for AI to exhibit Statistical Creativity relative to real human creators?
• RQ3 How can autoregressive models and prompting setups be used to practically evaluate AI creativity?
• RQ4 What are the theoretical and practical implications of incorporating generative conditions into AI training?
• RQ5 How does prompt-contextualization affect measures of AI creativity in LLMs?

Key findings

• A theoretical result showing AI can be as creative as humans if it can fit a sufficient amount of conditional data produced by human creators.
• Introduction of Relative Creativity (relative benchmark to a hypothetical human) and Statistical Creativity (data-driven similarity to real human creators).
• Demonstration that for autoregressive models, creativity can be measured via log-likelihood-based metrics (E1) with a sample-size bound (Theorem 2).
• Extension to prompt-based contexts (Corollaries 1 and 2) showing how LLMs’ creativity can be evaluated in prompting setups.
• Training guidance emphasizing inclusion of generation conditions and processes to foster creativity (Remark 7).
• Corollary 3 connects creativity loss minimization to generalization bounds, linking theoretical creativity to practical training outcomes.