[Paper Review] Model Exploration Using OpenMOLE - a workflow engine for large scale distributed design of experiments and parameter tuning

Romain Reuillon, Mathieu Leclaire+1 more|arXiv (Cornell University)|Jun 12, 2015

Scientific Computing and Data Management11 references3 citations

TL;DR

This paper presents OpenMOLE, a scientific workflow engine that enables large-scale distributed design of experiments and parameter tuning using a high-level Scala-based DSL. It demonstrates efficient multi-objective optimization of an agent-based ant simulation model via NSGA-II and island-model genetic algorithms, achieving 200,000 individual evaluations in one hour on the European Grid Infrastructure (EGI).

ABSTRACT

OpenMOLE is a scientific workflow engine with a strong emphasis on workload distribution. Workflows are designed using a high level Domain Specific Language (DSL) built on top of Scala. It exposes natural parallelism constructs to easily delegate the workload resulting from a workflow to a wide range of distributed computing environments. In this work, we briefly expose the strong assets of OpenMOLE and demonstrate its efficiency at exploring the parameter set of an agent simulation model. We perform a multi-objective optimisation on this model using computationally expensive Genetic Algorithms (GA). OpenMOLE hides the complexity of designing such an experiment thanks to its DSL, and transparently distributes the optimisation process. The example shows how an initialisation of the GA with a population of 200,000 individuals can be evaluated in one hour on the European Grid Infrastructure.

Motivation & Objective

• Address the challenge of exploring high-dimensional parameter spaces in complex simulation models.
• Overcome the limitations of single-machine execution for computationally expensive experiments.
• Enable transparent distribution of parameter tuning workloads across heterogeneous high-performance computing environments.
• Provide a scalable, reproducible, and user-friendly framework for scientific workflows in simulation research.
• Demonstrate the feasibility and efficiency of large-scale evolutionary computation using OpenMOLE on production grid infrastructures.

Proposed method

• Employ a Domain Specific Language (DSL) built on Scala to express scientific workflows and experimental designs.
• Utilize the GridScale library to interface with diverse distributed environments, including SSH, cluster managers, and the EGI grid.
• Implement multi-objective optimization using the NSGA-II algorithm with configurable inputs, objectives, and reevaluation for robustness.
• Apply the island model of evolutionary computation to distribute parallel populations across remote nodes, improving scalability.
• Integrate hooks for saving Pareto frontiers and monitoring progress during execution.
• Leverage the EGI environment with defined memory and wall time constraints to execute 2,000 concurrent islands in parallel.

Experimental results

Research questions

• RQ1 Can OpenMOLE effectively abstract the complexity of deploying large-scale experiments on distributed computing infrastructures?
• RQ2 How efficiently can OpenMOLE scale parameter tuning workloads using the island model across a grid environment?
• RQ3 To what extent does OpenMOLE reduce the deployment and management overhead for scientific workflows in distributed execution?
• RQ4 Can OpenMOLE support complex, multi-objective optimization of agent-based models with reproducible and scalable execution?
• RQ5 What performance gains are achievable when distributing a genetic algorithm with 200,000 individuals across a production grid?

Key findings

• OpenMOLE successfully executed a multi-objective optimization of an agent-based ant model with 200,000 individuals in one hour on the European Grid Infrastructure.
• The island model enabled 2,000 concurrent evolutionary processes, each with 50 individuals, achieving 200,000 evaluations across the entire workflow.
• The NSGA-II algorithm with reevaluation (1%) produced robust, statistically sound results by eliminating overestimated individuals.
• The DSL and workflow composition model allowed complex experimental designs to be expressed concisely and portably across platforms.
• The system achieved near-linear speedup in execution time relative to single-machine performance, approaching the ideal of distributed execution.
• OpenMOLE’s zero-deployment approach and on-the-fly software provisioning enabled reliable execution across heterogeneous environments without local installation.