[Paper Review] EvoTorch: Scalable Evolutionary Computation in Python

Nihat Engin Toklu, Timothy Atkinson+3 more|arXiv (Cornell University)|Feb 24, 2023

Evolutionary Algorithms and Applications4 citations

TL;DR

EvoTorch is a PyTorch-based evolutionary computation library that enables scalable, GPU-accelerated optimization for high-dimensional problems by integrating seamless hardware acceleration and distributed parallelization. It supports both continuous and discrete optimization, including complex structures like variable-length lists and genetic programming, achieving near-flat wall-clock times on GPU even at population sizes up to 100,000, outperforming CPU-based methods by over 10x.

ABSTRACT

Evolutionary computation is an important component within various fields such as artificial intelligence research, reinforcement learning, robotics, industrial automation and/or optimization, engineering design, etc. Considering the increasing computational demands and the dimensionalities of modern optimization problems, the requirement for scalable, re-usable, and practical evolutionary algorithm implementations has been growing. To address this requirement, we present EvoTorch: an evolutionary computation library designed to work with high-dimensional optimization problems, with GPU support and with high parallelization capabilities. EvoTorch is based on and seamlessly works with the PyTorch library, and therefore, allows the users to define their optimization problems using a well-known API.

Motivation & Objective

• To address the growing need for scalable, reusable, and high-performance evolutionary computation in modern AI and optimization tasks.
• To enable efficient GPU acceleration for evolutionary algorithms by building natively on PyTorch, avoiding manual tensor transfers.
• To support diverse optimization problems, including discrete and structured representations like lists, stacks, and dictionaries, beyond standard continuous vectors.
• To provide seamless integration with distributed computing via Ray, enabling large-scale experiments across multi-node clusters.
• To facilitate neuroevolution and black-box optimization with state-of-the-art algorithms such as PGPE and XNES.

Proposed method

• EvoTorch represents solutions as PyTorch tensors for hardware-accelerated operations, enabling vectorized fitness evaluation on GPU.
• It introduces custom, differentiable data structures like CList, CStack, and CQueue for handling variable-length sequences in a batched, GPU-parallelizable manner.
• The library supports multiple parallelization modes: GPU-based across multiple GPUs, CPU-based across multiple cores, and distributed across clusters using Ray.
• Fitness functions can be defined using standard PyTorch operations, with automatic differentiation support where applicable.
• It integrates with established PyTorch-based ecosystems (e.g., PyTorch Geometric, Detectron2) for immediate compatibility with existing ML pipelines.
• Evolutionary algorithms such as PGPE and XNES are natively implemented and optimized for GPU execution.

Experimental results

Research questions

• RQ1 Can evolutionary computation be efficiently scaled using GPU acceleration within the PyTorch ecosystem?
• RQ2 How does GPU-based parallelization affect wall-clock time for large population sizes in discrete optimization problems like genetic programming?
• RQ3 To what extent can PyTorch-based data structures enable vectorized, hardware-accelerated evaluation of algorithmic or sequential fitness functions?
• RQ4 Can a PyTorch-native EC library outperform traditional CPU-based libraries in terms of scalability and performance for high-dimensional problems?
• RQ5 How does the integration of Ray enable distributed evolutionary computation across multi-node clusters while maintaining compatibility with PyTorch?

Key findings

• EvoTorch achieves near-constant wall-clock time for fitness evaluation across populations up to 30,000 on a single GPU, demonstrating strong scalability.
• For population sizes of 100,000, the GPU-based implementation took over 10 times less time than the CPU-based implementation on a 40-thread CPU.
• The use of CList and other vectorized structures enabled efficient, loop-free manipulation of variable-length lists across batches, improving code clarity and performance.
• The library successfully supports complex discrete problems such as genetic programming, where fitness evaluation involves sequential, algorithmic interpretation of programs.
• EvoTorch enables end-to-end GPU acceleration for both fitness evaluation and evolutionary algorithm operations, unlike CPU-only or manually transferred approaches.
• The integration with Ray allows for seamless scaling across multiple machines, supporting large-scale distributed evolutionary experiments.