[Paper Review] A Survey on Distributed Evolutionary Computation

Wei–Neng Chen, Feng-Feng Wei+3 more|arXiv (Cornell University)|Apr 12, 2023

Metaheuristic Optimization Algorithms Research4 citations

TL;DR

This paper presents a comprehensive survey on distributed evolutionary computation (DEC), proposing a novel taxonomy from algorithmic design to implementation. It systematically reviews DEC for centralized optimization (via parallel processing) and distributed optimization (with partial information), classifying the latter into dimension-, data-, and objective-distributed problems, and identifies key challenges and future research directions in the field.

ABSTRACT

The rapid development of parallel and distributed computing paradigms has brought about great revolution in computing. Thanks to the intrinsic parallelism of evolutionary computation (EC), it is natural to implement EC on parallel and distributed computing systems. On the one hand, the computing power provided by parallel computing systems can significantly improve the efficiency and scalability of EC. On the other hand, data are collected and processed in a distributed manner, which brings a novel development direction and new challenges to EC. In this paper, we intend to give a systematic review on distributed EC (DEC). First, a new taxonomy for DEC is proposed from top design mechanism to bottom implementation mechanism. Based on this taxonomy, existing studies on DEC are reviewed in terms of purpose, parallel structure of the algorithm, parallel model for implementation, and the implementation environment. Second, we clarify two major purposes of DEC, i.e., improving efficiency through parallel processing for centralized optimization and cooperating distributed individuals/sub-populations with partial information to perform distributed optimization. Third, noting that the latter purpose of DEC is an emerging and attractive trend for EC with the booming of spatially distributed paradigms, this paper gives a systematic definition of the distributed optimization and classifies it into dimension distributed-, data distributed-, and objective distributed-optimization problems. Formal formulations for these problems are provided and various DEC studies on these problems are reviewed. We also discuss challenges and potential research directions, aiming to enlighten the design of DEC and pave the way for future developments.

Motivation & Objective

• To provide a systematic review of distributed evolutionary computation (DEC) in response to advances in parallel and distributed computing.
• To clarify the two primary purposes of DEC: improving efficiency through parallel processing and enabling cooperation among distributed individuals with partial information.
• To formalize and classify distributed optimization problems into dimension-, data-, and objective-distributed categories with mathematical formulations.
• To identify open challenges and suggest promising research directions for future development in DEC.

Proposed method

• Proposes a hierarchical taxonomy for DEC, ranging from top-level design mechanisms to low-level implementation techniques.
• Classifies DEC studies based on purpose, algorithmic parallel structure, implementation model, and deployment environment.
• Introduces formal mathematical formulations for three types of distributed optimization: dimension-, data-, and objective-distributed problems.
• Reviews existing DEC approaches across different problem types, emphasizing algorithmic strategies and communication patterns.
• Analyzes the trade-offs between convergence speed, solution quality, and communication overhead in distributed EC systems.
• Highlights emerging trends in spatially distributed EC and the role of partial information sharing in decentralized optimization.

Experimental results

Research questions

• RQ1 How can evolutionary computation be effectively scaled using distributed and parallel computing infrastructures?
• RQ2 What are the key distinctions between centralized optimization via parallel EC and true distributed optimization with partial information?
• RQ3 How can distributed optimization problems be formally categorized and mathematically modeled in the context of evolutionary computation?
• RQ4 What are the major challenges in designing efficient and scalable distributed evolutionary algorithms?
• RQ5 What future research directions are most promising for advancing distributed evolutionary computation?

Key findings

• The paper establishes a formal classification of distributed optimization into dimension-, data-, and objective-distributed problems, each with distinct mathematical formulations.
• It identifies that distributed optimization is an emerging and increasingly important trend due to the rise of spatially distributed data and systems.
• The survey reveals that communication overhead and load balancing are critical challenges in distributed evolutionary computation, especially in large-scale deployments.
• Existing DEC approaches show improved scalability and efficiency in centralized settings, but distributed coordination remains a major open issue.
• The integration of partial information sharing and decentralized decision-making is shown to be essential for solving real-world distributed optimization problems.
• The authors conclude that future research should focus on adaptive communication strategies, fault tolerance, and hybrid models combining centralized and distributed paradigms.