[Paper Review] Game Level Clustering and Generation using Gaussian Mixture VAEs

Zhihan Yang, Anurag Sarkar+1 more|arXiv (Cornell University)|Aug 22, 2020

Artificial Intelligence in Games4 citations

TL;DR

This paper proposes using Gaussian Mixture Variational Autoencoders (GMVAEs) to unsupervisedly cluster and generate game level chunks from Super Mario Bros., Kid Icarus, and Mega Man. By modeling the latent space with a mixture of Gaussians, GMVAEs discover distinct design patterns without labels and generate new levels with desired characteristics, outperforming baseline VAE-GMM approaches in clustering disentanglement and structural coherence.

ABSTRACT

Variational autoencoders (VAEs) have been shown to be able to generate game levels but require manual exploration of the learned latent space to generate outputs with desired attributes. While conditional VAEs address this by allowing generation to be conditioned on labels, such labels have to be provided during training and thus require prior knowledge which may not always be available. In this paper, we apply Gaussian Mixture VAEs (GMVAEs), a variant of the VAE which imposes a mixture of Gaussians (GM) on the latent space, unlike regular VAEs which impose a unimodal Gaussian. This allows GMVAEs to cluster levels in an unsupervised manner using the components of the GM and then generate new levels using the learned components. We demonstrate our approach with levels from Super Mario Bros., Kid Icarus and Mega Man. Our results show that the learned components discover and cluster level structures and patterns and can be used to generate levels with desired characteristics.

Motivation & Objective

• To address the limitation of conditional VAEs requiring labeled data for controllable generation by enabling unsupervised clustering of game level features.
• To explore whether GMVAEs can discover meaningful, semantically coherent clusters of game level chunks without prior knowledge of design patterns.
• To evaluate the quality of generated levels in terms of structural coherence, disentanglement, and playability.
• To compare GMVAE performance against a VAE-GMM baseline in clustering effectiveness and generation fidelity.
• To investigate the potential of GMVAEs as an exploratory tool for game designers to discover hidden structures in level design.

Proposed method

• The GMVAE model uses a k-component Gaussian Mixture (GM) prior over the latent space, allowing each component to learn a distinct cluster of level chunks.
• The encoder maps each 16x16 game chunk into a latent distribution parameterized by mean and variance, and a latent vector is sampled from this distribution.
• The decoder reconstructs the input chunk from the sampled latent vector, minimizing both reconstruction loss (binary cross-entropy) and KL divergence from the GM prior.
• The model is trained end-to-end using backpropagation with the reparameterization trick, enabling differentiable sampling from the latent distribution.
• Clustering performance is evaluated by training a multi-layer perceptron (MLP) on generated samples from each component to assess disentanglement.
• Playability is measured using an adapted A* search algorithm on aggregated chunks from each component, with scores reported per game.

Experimental results

Research questions

• RQ1 Can GMVAEs unsupervisedly discover meaningful clusters of game level chunks based on design patterns such as presence of specific elements or structural layouts?
• RQ2 How does the clustering performance of GMVAEs compare to a VAE-GMM baseline in terms of component disentanglement and structural coherence?
• RQ3 Can the learned components of a GMVAE generate new level chunks with consistent and desired characteristics?
• RQ4 To what extent do generated chunks maintain playability, especially when aggregated into larger levels?
• RQ5 Do the clusters discovered by GMVAEs align with human-perceived level types or design categories?

Key findings

• GMVAEs achieved at least 99.4% playability for generated chunks from Super Mario Bros. and 79.2% for Kid Icarus, indicating high structural quality.
• The proportion of components that were ≥70% disentangled remained stable and high for GMVAEs, while dropping significantly for the VAE-GMM baseline as the number of components increased.
• For both SMB and MM, GMVAE performance at 2 components was poor, likely due to overfitting multiple design patterns into each component, reducing discriminability.
• The GMVAE outperformed the VAE-GMM baseline in clustering disentanglement, as evidenced by higher and more stable proportions of well-separated components across varying numbers of components.
• Each GMVAE component learned to generate chunks with consistent design features, such as specific enemy placements or platforming patterns, enabling targeted generation.
• The latent space of GMVAEs allows for potential interpolation between clusters, though such interpolations may lie in unexplored regions and could yield novel level structures.