[Paper Review] Cumulative Step-size Adaptation on Linear Functions

Chotard, Alexandre, Auger, Anne+1 more|HAL (Le Centre pour la Communication Scientifique Directe)|Dec 1, 2012

Markov Chains and Monte Carlo Methods11 references4 citations

TL;DR

This paper analyzes the (1,λ)-CSA-ES on linear functions using Markov chain theory, showing that step-size adaptation via cumulative path leads to geometrically fast divergence of the step size for λ ≥ 3 or λ = 2 with cumulation (c < 1). The key contribution is a rigorous variance analysis proving that choosing c = 1/n^α with α > 1/3 ensures the signal-to-noise ratio of step-size increments becomes negligible as dimension increases, guaranteeing algorithmic stability.

ABSTRACT

The CSA-ES is an Evolution Strategy with Cumulative Step size Adaptation, where the step size is adapted measuring the length of a so-called cumulative path. The cumulative path is a combination of the previous steps realized by the algorithm, where the importance of each step decreases with time. This article studies the CSA-ES on composites of strictly increasing functions with affine linear functions through the investigation of its underlying Markov chains. Rigorous results on the change and the variation of the step size are derived with and without cumulation. The step-size diverges geometrically fast in most cases. Furthermore, the influence of the cumulation parameter is studied.

Motivation & Objective

• To understand the long-term behavior of step-size adaptation in the (1,λ)-CSA-ES on linear functions.
• To rigorously analyze the dynamics of the logarithmic step-size, ln(σ_t), under both cumulation and non-cumulation regimes.
• To quantify the variance of step-size increments and determine conditions under which the signal-to-noise ratio of step-size adaptation becomes negligible.
• To establish theoretical conditions on the cumulation parameter c that ensure stable and convergent behavior in high-dimensional spaces.

Proposed method

• Models the (1,λ)-CSA-ES as a Markov chain on the logarithmic step-size and cumulative path length.
• Derives the limiting distribution of ln(σ_t)/t under the assumption of i.i.d. standard normal perturbations in the selection process.
• Uses the cumulative path update rule: p_{t+1} = (1−c)p_t + √(c(2−c)) ξ_t^*, where ξ_t^* is the best-selected search vector.
• Applies variance decomposition to ln(σ_{t+1}/σ_t), expressing it in terms of moments of the path component [p_{t+1}]_1.
• Analyzes the asymptotic behavior of the standard deviation of step-size increments relative to their expectation, particularly as dimension n → ∞.
• Considers the choice c = 1/(1 + n^α) to study the scaling of relative variance with dimension and identifies critical α values.

Experimental results

Research questions

• RQ1 Does the step size in the (1,λ)-CSA-ES diverge geometrically fast on linear functions?
• RQ2 How does the cumulation parameter c affect the stability and variance of step-size adaptation?
• RQ3 What is the asymptotic behavior of the variance of the logarithmic step-size increment as the dimension n increases?
• RQ4 For which values of c does the signal-to-noise ratio of step-size adaptation tend to zero in high dimensions?
• RQ5 How does the choice of λ (especially λ = 2 vs. λ ≥ 3) influence the long-term behavior of the step size?

Key findings

• The step size diverges geometrically fast in almost all cases, except when λ = 2 and c = 1, where the logarithmic step size performs a random walk.
• For λ ≥ 3 or λ = 2 with c < 1, the logarithmic step size grows linearly almost surely, indicating strong divergence.
• The variance of the logarithmic step-size increment scales as √((n^{2α} + n)/n^{3α}) times the expected increment when c = 1/(1 + n^α).
• When α > 1/3, the relative standard deviation of the step-size increment tends to zero as n → ∞, ensuring high stability.
• The critical value α = 1/3 marks the threshold where the relative standard deviation converges to a constant, while α < 1/3 leads to divergence of the relative variance.
• The choice c = 1/√n is confirmed as stable, as it corresponds to α = 1/2 > 1/3, ensuring negligible relative variance in high dimensions.