[Paper Review] The CMA Evolution Strategy: A Tutorial

Nikolaus Hansen|arXiv (Cornell University)|Apr 4, 2016

Evolutionary Algorithms and Applications640 citations

TL;DR

This paper presents the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) as a stochastic, real-parameter optimization method, detailing its sampling, selection, covariance adaptation, and step-size control mechanisms. It also provides algorithmic summaries, implementational notes, and MATLAB code references.

ABSTRACT

This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized, method for real-parameter (continuous domain) optimization of non-linear, non-convex functions. We try to motivate and derive the algorithm from intuitive concepts and from requirements of non-linear, non-convex search in continuous domain.

Motivation & Objective

• Introduce CMA-ES as a stochastic method for real-parameter optimization in continuous domains.
• Explain the sampling process from a multivariate normal distribution and how the mean, covariance, and step-size evolve.
• Derive and explain rank-based covariance updates (rank-one and rank-e6 updates) and cumulation via evolution paths.
• Present practical considerations, algorithm summaries, and MATLAB source references for implementation.

Proposed method

• Model Search: Samples lambda offspring from a multivariate normal distribution with mean m and covariance C.
• Selection and Recombination: Update the mean using recombination weights w_i from the top mu offspring.
• Covariance Adaptation: Update C using rank-one and rank-6 updates and cumulation through evolution paths p and y.
• Step-Size Control: Adapt the global step-size sigma via a cumulation mechanism and damping parameter d_sigma.
• Eigen decomposition: Use C = B D^2 B^T to interpret and implement updates efficiently (B orthogonal, D diagonal).
• Algorithm Summary: Appendix A provides a concise description of CMA-ES; Appendix B discusses implementational concerns; Appendix C provides MATLAB code.

Experimental results

Research questions

• RQ1 How can the covariance matrix be effectively adapted during evolution to guide search in continuous spaces?
• RQ2 What are robust strategies for updating the mean and step-size to balance exploration and exploitation?
• RQ3 How can cumulation paths be used to stabilize updates and improve optimization performance?
• RQ4 What are practical considerations for implementing CMA-ES, including numerical and boundary handling?
• RQ5 What is the theoretical relationship between CMA-ES parameters and the geometry of the search distribution?

Key findings

• CMA-ES models the search distribution as a multivariate normal with evolving mean and covariance.
• Covariance updates combine rank-one and rank-mu updates to capture curvature information.
• Cumulation via evolution paths stabilizes updates and improves performance on non-linear, non-convex landscapes.
• The algorithm uses eigen decomposition of C to interpret and implement sampling and updates efficiently.
• The paper provides a complete algorithmic summary and MATLAB reference for practical implementation.