[Paper Review] General Upper Bounds on the Running Time of Parallel Evolutionary Algorithms

Jörg Lässig, Dirk Sudholt|arXiv (Cornell University)|Jun 15, 2012

Metaheuristic Optimization Algorithms Research29 references4 citations

TL;DR

This paper introduces a general method for deriving upper bounds on the expected running time of parallel evolutionary algorithms with spatially structured populations using the fitness-level method. It shows that denser migration topologies like complete graphs yield significantly higher speedups—up to order $ n^{k/2} $ for Jump functions—while maintaining asymptotically optimal function evaluation counts, with explicit population size recommendations for optimal performance.

ABSTRACT

We present a new method for analyzing the running time of parallel evolutionary algorithms with spatially structured populations. Based on the fitness-level method, it yields upper bounds on the expected parallel running time. This allows to rigorously estimate the speedup gained by parallelization. Tailored results are given for common migration topologies: ring graphs, torus graphs, hypercubes, and the complete graph. Example applications for pseudo-Boolean optimization show that our method is easy to apply and that it gives powerful results. In our examples the possible speedup increases with the density of the topology. Surprisingly, even sparse topologies like ring graphs lead to a significant speedup for many functions while not increasing the total number of function evaluations by more than a constant factor. We also identify which number of processors yield asymptotically optimal speedups, thus giving hints on how to parametrize parallel evolutionary algorithms.

Motivation & Objective

• To provide a rigorous theoretical framework for analyzing the running time of parallel evolutionary algorithms with spatially structured populations.
• To estimate the speedup gained by parallelization and the total communication effort in terms of migrated individuals.
• To identify optimal population sizes that yield asymptotically optimal speedups while minimizing function evaluations.
• To compare the performance of different migration topologies—ring, torus, hypercube, complete graph—on standard pseudo-boolean functions.
• To offer practical guidelines for parametrizing parallel EAs based on theoretical bounds.

Proposed method

• Adapts the fitness-level method to parallel evolutionary algorithms by partitioning the search space into fitness-level sets ordered by fitness.
• Derives a general upper bound on the expected parallel running time based on the minimum transmission probability $ p $ of individuals from the best fitness level to neighboring islands.
• Tailors the general bound to specific topologies: ring, torus, hypercube, and complete graph, using their respective connectivity properties.
• Uses the concept of 'effective waiting time' reduction—where the expected time to improve is reduced by a factor depending on topology density (e.g., square root for rings, cube root for tori, logarithmic for hypercubes, constant for complete graphs).
• Applies the method to standard functions (OneMax, LO, Jump_k) to derive explicit bounds on parallel running time and communication cost.
• Establishes that optimal population size $ \mu $ can be chosen such that total function evaluations remain asymptotically unchanged, even with significant speedup.

Experimental results

Research questions

• RQ1 How does the choice of migration topology affect the expected running time of parallel evolutionary algorithms?
• RQ2 What is the maximum speedup achievable through parallelization, and how does it scale with topology density?
• RQ3 For which population sizes $ \mu $ is the speedup asymptotically optimal, and how does this depend on the function class?
• RQ4 Can the total number of function evaluations be kept asymptotically unchanged while achieving significant speedup?
• RQ5 How do communication costs scale with topology and population size, and what trade-offs exist between time and communication?

Key findings

• For OneMax, the ring topology reduces the expected waiting time per fitness level to its square root, yielding a $ \log n $ speedup when $ p = \Omega(1) $.
• For unimodal functions like LO, the ring topology achieves a $ \sqrt{n} $ speedup, and the torus topology reduces the waiting time to its cube root, yielding $ O(n^{4/3}) $ parallel running time.
• On $ \textup{Jump}_k $ functions with $ 3 \leq k = O(n/\log n) $, the speedup reaches order $ n^{k/2} $, with the complete graph achieving $ O(n) $ parallel running time.
• The complete graph achieves the smallest parallel running time bound of $ O(n) $, but at the cost of $ O(n^3) $ communication effort, while the hypercube offers a good balance with $ O(n \log n) $ communication and $ O(n) $ time.
• For LO, the torus graph minimizes communication cost at $ O(n^2) $, but has a higher time bound of $ O(n^{4/3}) $, while the hypercube provides a better trade-off in both metrics.
• Population sizes $ \mu = O(\log n) $ are sufficient for linear speedup on OneMax with $ p = \Omega(1) $, while $ \mu = O(\sqrt{n}) $ is optimal for rings on LO, and $ \mu = \Theta(n) $ is required for linear speedup on LO with complete graphs.