[Paper Review] Procedural Content Generation via Generative Artificial Intelligence

Xinyu Mao, Wanli Yu+2 more|arXiv (Cornell University)|Jul 12, 2024

Artificial Intelligence in Law5 citations

TL;DR

This survey reviews how generative AI, particularly GANs and related models, is applied to procedural content generation in video games, covering environments, assets, narratives, and audio, and discusses data limitations.

ABSTRACT

The attempt to utilize machine learning in PCG has been made in the past. In this survey paper, we investigate how generative artificial intelligence (AI), which saw a significant increase in interest in the mid-2010s, is being used for PCG. We review applications of generative AI for the creation of various types of content, including terrains, items, and even storylines. While generative AI is effective for PCG, one significant issues it faces is that building high-performance generative AI requires vast amounts of training data. Because content generally highly customized, domain-specific training data is scarce, and straightforward approaches to generative AI models may not work well. For PCG research to advance further, issues related to limited training data must be overcome. Thus, we also give special consideration to research that addresses the challenges posed by limited training data.

Motivation & Objective

• Summarize the historical and current use of AI in PCG, from rule-based and classical ML to generative AI methods.
• Highlight how generative AI is applied across game content types such as environments, assets, narratives, and audio.
• Discuss data scarcity challenges in domain-specific PCG and approaches to mitigate training data limitations.
• Provide insights and future directions for integrating generative AI into game development workflows.

Proposed method

• Classify content types in games and relate generation objectives to playability, coherence, and aesthetics.
• Describe generative AI concepts and models, including discriminative vs. generative frameworks and GANs.
• Review GAN-based PCG work across 2D levels, 3D terrains, sprites, characters, narratives, and audio.
• Summarize example studies that train on limited data or use multi-step and conditional GAN architectures.
• Discuss data- and model-centric challenges and potential research directions in PCG with generative AI.

Experimental results

Research questions

• RQ1 What are the main generative AI approaches used for procedural content generation in video games?
• RQ2 How are GANs and related generative models applied to different PCG content types (environments, assets, narratives, audio)?
• RQ3 What are the practical challenges, especially data scarcity, in training generative PCG systems, and how are they addressed?
• RQ4 What future directions can advance the integration of generative AI into PCG workflows?

Key findings

• GANs and conditional GANs can generate playable game levels from image- or feature-based representations.
• Several studies demonstrate that latent space exploration or multi-step GAN pipelines can control level properties and solvability.
• GAN-based terrain and asset generation can leverage real-world data or minimal examples to achieve realistic results.
• Generative AI enables diverse and dynamic narrative and character content through AI-driven agents and memory/planning mechanisms.
• Datasets and data efficiency remain a central challenge, motivating methods like conditioning, transfer learning, and architecture innovations to work with limited data.