[Paper Review] Illuminating search spaces by mapping elites

Jean-Baptiste Mouret, Jeff Clune|arXiv (Cornell University)|Apr 20, 2015

Reinforcement Learning in Robotics59 references406 citations

TL;DR

This paper introduces MAP-Elites, a novel evolutionary algorithm that maps high-performing solutions across a user-defined feature space, illuminating trade-offs between performance and design attributes. By systematically exploring and archiving elite solutions, it discovers a diverse set of high-quality solutions while outperforming traditional optimizers in global performance, even when searching for a single best solution.

ABSTRACT

Many fields use search algorithms, which automatically explore a search space to find high-performing solutions: chemists search through the space of molecules to discover new drugs; engineers search for stronger, cheaper, safer designs, scientists search for models that best explain data, etc. The goal of search algorithms has traditionally been to return the single highest-performing solution in a search space. Here we describe a new, fundamentally different type of algorithm that is more useful because it provides a holistic view of how high-performing solutions are distributed throughout a search space. It creates a map of high-performing solutions at each point in a space defined by dimensions of variation that a user gets to choose. This Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) algorithm illuminates search spaces, allowing researchers to understand how interesting attributes of solutions combine to affect performance, either positively or, equally of interest, negatively. For example, a drug company may wish to understand how performance changes as the size of molecules and their cost-to-produce vary. MAP-Elites produces a large diversity of high-performing, yet qualitatively different solutions, which can be more helpful than a single, high-performing solution. Interestingly, because MAP-Elites explores more of the search space, it also tends to find a better overall solution than state-of-the-art search algorithms. We demonstrate the benefits of this new algorithm in three different problem domains ranging from producing modular neural networks to designing simulated and real soft robots. Because MAP- Elites (1) illuminates the relationship between performance and dimensions of interest in solutions, (2) returns a set of high-performing, yet diverse solutions, and (3) improves finding a single, best solution, it will advance science and engineering.

Motivation & Objective

• To address the limitation of traditional search algorithms that return only a single or few high-performing solutions, often missing critical trade-offs in design space.
• To develop a method that provides a holistic, visual map of how performance varies across dimensions of interest, such as size, cost, or modularity.
• To improve global optimization performance by exploring more of the search space, thereby increasing the likelihood of finding better overall solutions.
• To enable researchers and engineers to understand not just the best solution, but how performance emerges from combinations of design features.
• To provide a citable, foundational reference for a widely used algorithm in evolutionary computation and optimization.

Proposed method

• MAP-Elites uses a multi-dimensional feature space defined by user-selected attributes (e.g., size, cost, modularity) to organize and archive high-performing solutions.
• The algorithm maintains a grid (map) where each cell corresponds to a unique combination of feature values, and stores the highest-performing solution found for each cell.
• It employs a hierarchical, parallelized evolutionary algorithm to explore the search space, generating new solutions through mutation and selection, with fitness evaluated via simulation or experiment.
• The resolution of the map is adaptively increased over time, starting from a coarse grid and refining to higher resolution as the search progresses.
• Solutions are mapped to cells based on their feature values, and only the highest-performing solution per cell is retained, ensuring elite preservation.
• The algorithm uses a batch-based update strategy, where multiple solutions are evaluated per iteration, and the map is updated iteratively to cover more of the feature space.

Experimental results

Research questions

• RQ1 Can a search algorithm provide a comprehensive, visual map of performance across user-defined design features, revealing trade-offs and relationships between attributes and performance?
• RQ2 Does exploring a broader region of the search space through feature-based mapping lead to better overall solutions than traditional optimization algorithms?
• RQ3 To what extent does MAP-Elites discover diverse, high-performing solutions that are qualitatively different, compared to algorithms that return only a single best solution?
• RQ4 How does the algorithm’s performance compare to state-of-the-art methods in terms of global reliability, precision, and coverage of the feature space?
• RQ5 Can MAP-Elites be effectively applied across diverse domains such as neural network design, soft robotics, and physical robot control?

Key findings

• MAP-Elites achieved higher global performance than state-of-the-art optimization algorithms in all tested domains, including soft robotics and neural network design.
• The algorithm discovered a diverse set of high-performing solutions across the feature space, revealing performance trade-offs that were not apparent with single-solution optimizers.
• In the retina problem, MAP-Elites mapped performance across connection cost and network modularity, identifying high-performing configurations that balanced these competing features.
• For soft robot design, the algorithm explored combinations of bone percentage and voxel fill, producing a map of locomotion performance that revealed optimal design regions.
• In the physical arm experiment, MAP-Elites outperformed random sampling and grid search in both coverage and fitness, finding the highest-reaching configuration.
• The algorithm demonstrated superior global reliability and precision, with performance normalized against the best-known solution per cell across all runs.