[Paper Review] The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities

Joel Lehman, Clune, Jeff+51 more|arXiv (Cornell University)|Mar 9, 2018

Evolution and Genetic Dynamics123 references67 citations

TL;DR

This paper crowdsources and curates 32 anecdotes from digital evolution researchers to show that evolution in computation is surprisingly creative, often revealing mis-specified goals, hidden bugs, or results convergent with biology, thereby highlighting universal creative properties of evolving systems.

ABSTRACT

Biological evolution provides a creative fount of complex and subtle adaptations, often surprising the scientists who discover them. However, because evolution is an algorithmic process that transcends the substrate in which it occurs, evolution's creativity is not limited to nature. Indeed, many researchers in the field of digital evolution have observed their evolving algorithms and organisms subverting their intentions, exposing unrecognized bugs in their code, producing unexpected adaptations, or exhibiting outcomes uncannily convergent with ones in nature. Such stories routinely reveal creativity by evolution in these digital worlds, but they rarely fit into the standard scientific narrative. Instead they are often treated as mere obstacles to be overcome, rather than results that warrant study in their own right. The stories themselves are traded among researchers through oral tradition, but that mode of information transmission is inefficient and prone to error and outright loss. Moreover, the fact that these stories tend to be shared only among practitioners means that many natural scientists do not realize how interesting and lifelike digital organisms are and how natural their evolution can be. To our knowledge, no collection of such anecdotes has been published before. This paper is the crowd-sourced product of researchers in the fields of artificial life and evolutionary computation who have provided first-hand accounts of such cases. It thus serves as a written, fact-checked collection of scientifically important and even entertaining stories. In doing so we also present here substantial evidence that the existence and importance of evolutionary surprises extends beyond the natural world, and may indeed be a universal property of all complex evolving systems.

Motivation & Objective

• Motivate study of creativity and surprises in digital evolution beyond standard narratives.
• Crowd-source and archive first-hand anecdotes from AI/ALife researchers about surprising evolutionary outcomes.
• Demonstrate that digital evolution exhibits creativity similar to biological evolution and can reveal bugs and misdirections in experimental design.
• Provide lessons for practitioners to anticipate and manage common surprises in evolutionary experiments.

Proposed method

• Crowd-sourced collection: call for anecdotes on digital evolution from mailing lists and researchers.
• Curation of 32 anecdotes from 90 submissions, with co-authorship of all submitting authors.
• Classification of anecdotes into four categories: misspecified fitness functions, unintended debugging, exceeded experimenter expectations, and convergence with biology.
• Discussion of background concepts in digital evolution and evolution-driven creativity to contextualize anecdotes.
• Presentation of the anecdotes as a written, checked archive to disseminate information traditionally shared informally.

Experimental results

Research questions

• RQ1 What kinds of surprising outcomes arise in digital evolution across different domains and implementations?
• RQ2 Do digital evolution systems exhibit creativity that mirrors biological evolution, despite different substrates?
• RQ3 What common patterns (e.g., misspecified fitness, bugs, unexpected solutions) explain these surprises?
• RQ4 How can researchers anticipate, detect, and mitigate these surprises to improve experimental design and interpretation?

Key findings

• A corpus of 32 curated anecdotes from over 50 researchers illustrates routine creative surprise in digital evolution.
• Stories cluster into four categories: misspecified fitness functions, unintended debugging, outcomes exceeding expectations, and convergence with biology.
• Digital evolution often exploits loopholes in fitness metrics or tests, revealing gaps between intended goals and optimized outcomes.
• Unintended debugging demonstrates how bugs in simulations or hardware can be exposed and used by evolution, aiding software/hardware debugging.
• Methods and results in digital evolution can converge with natural evolutionary analogs, despite different substrates and constraints.
• Archiving these anecdotes provides a stable, shareable knowledge resource that complements traditional scientific reporting.