[Paper Review] Digits that are not: Generating new types through deep neural nets

Cherti, Akın Kazakçıand Mehdi, Balázs Kégl|arXiv (Cornell University)|Jun 14, 2016

Artificial Intelligence in Games26 references3 citations

TL;DR

This paper proposes a knowledge-driven creativity framework using a deep generative autoencoder to produce novel, unseen digit types that are semantically coherent yet not belonging to any known digit class. By learning hierarchical features from handwritten digits, the model generates new symbols through compositional sampling, demonstrating how artificial agents can autonomously explore novelty based on internal knowledge rather than externally imposed value functions.

ABSTRACT

For an artificial creative agent, an essential driver of the search for novelty is a value function which is often provided by the system designer or users. We argue that an important barrier for progress in creativity research is the inability of these systems to develop their own notion of value for novelty. We propose a notion of knowledge-driven creativity that circumvent the need for an externally imposed value function, allowing the system to explore based on what it has learned from a set of referential objects. The concept is illustrated by a specific knowledge model provided by a deep generative autoencoder. Using the described system, we train a knowledge model on a set of digit images and we use the same model to build coherent sets of new digits that do not belong to known digit types.

Motivation & Objective

• To address the limitation of existing creative systems that rely on externally imposed value functions for novelty.
• To enable artificial agents to develop their own internal value system for novelty based on learned domain knowledge.
• To explore generative deep neural networks beyond known object types, focusing on creating new semantic types.
• To demonstrate that deep autoencoders can generate coherent, novel symbols that are not recognizable as standard digits but retain semantic consistency.
• To provide a foundation for studying how machines can autonomously drive creative exploration through self-constructed knowledge models.

Proposed method

• Training a deep generative autoencoder on a dataset of handwritten digits (0–9) to learn hierarchical, disentangled feature representations.
• Using the trained encoder to map input digits into a learned latent space representing abstract, compositional features.
• Applying a compositional sampling strategy in the latent space by combining features from different digits in non-canonical ways.
• Decoding the sampled latent vectors through the generative decoder to produce novel, visually coherent digit-like symbols.
• Ensuring semantic coherence by leveraging the autoencoder’s ability to reconstruct known digits, thus preserving structural and compositional rules.
• Evaluating novelty through perceptual analysis—results are unrecognizable as standard digits but appear as plausible, dreamlike symbols.

Experimental results

Research questions

• RQ1 Can a deep generative model produce novel object types that are not present in the training data, while preserving semantic consistency?
• RQ2 How can an artificial agent generate novelty without relying on an externally defined fitness or value function?
• RQ3 To what extent can a self-learned knowledge model guide the discovery of new types in a domain?
• RQ4 What role does the internal representation of a deep autoencoder play in enabling the generation of semantically plausible but novel symbols?
• RQ5 Can compositional sampling in the latent space of a generative model lead to systematic exploration of 'unknown' types beyond known categories?

Key findings

• The model successfully generated a set of novel digit-like symbols that are not recognizable as standard digits (0–9), yet appear coherent and plausible in form.
• The generated symbols exhibit structural and compositional regularities consistent with the domain, suggesting that the model has learned abstract, semantic-level features.
• The system demonstrates knowledge-driven creativity by exploring blind spots in the learned feature space, producing objects that are both novel and semantically plausible.
• The approach avoids reliance on external evaluation functions, instead enabling the system to explore novelty based on its own internal model of the domain.
• The results show that deep generative autoencoders can be used not only for reconstruction but also for systematic exploration of new types beyond known categories.
• The method provides a foundation for future systems that can build their own value functions through interaction with their environment, rather than relying on pre-specified criteria.