[论文解读] modOpt: A modular development environment and library for optimization algorithms

Anugrah Jo Joshy, John T. Hwang|arXiv (Cornell University)|Oct 16, 2024

Parallel Computing and Optimization Techniques被引用 5

一句话总结

modOpt 是一个开源的 Python 框架，能够从模块化组件构建和测试优化算法，提供对多种建模语言的接口，以及内置的教育算法、可视化和基准测试工具。

ABSTRACT

Recent advances in computing hardware and modeling software have given rise to new applications for numerical optimization. These new applications occasionally uncover bottlenecks in existing optimization algorithms and necessitate further specialization of the algorithms. However, such specialization requires expert knowledge of the underlying mathematical theory and the software implementation of existing algorithms. To address this challenge, we present modOpt, an open-source software framework that facilitates the construction of optimization algorithms from modules. The modular environment provided by modOpt enables developers to tailor an existing algorithm for a new application by only altering the relevant modules. modOpt is designed as a platform to support students and beginner developers in quickly learning and developing their own algorithms. With that aim, the entirety of the framework is written in Python, and it is well-documented, well-tested, and hosted open-source on GitHub. Several additional features are embedded into the framework to assist both beginner and advanced developers. In addition to providing stock modules, the framework also includes fully transparent implementations of pedagogical optimization algorithms in Python. To facilitate testing and benchmarking of new algorithms, the framework features built-in visualization and recording capabilities, interfaces to modeling frameworks such as OpenMDAO and CSDL, interfaces to general-purpose optimization algorithms such as SNOPT and SLSQP, an interface to the CUTEst test problem set, etc. In this paper, we present the underlying software architecture of modOpt, review its various features, discuss several educational and performance-oriented algorithms within modOpt, and present numerical studies illustrating its unique benefits.

研究动机与目标

• 通过将组件模块化（例如直线搜索、海森更新、准则函数）来降低开发优化算法的门槛。
• 提供一个统一、面向初学者的跨语言多优化器和模型的接口。
• 为初次接触优化的学生和研究人员提供测试、基准、可视化和学习的便利。
• 提供与现有建模框架(OpenMDAO, CSDL, CasADi, Jax)以及标准测试问题(CUTEst)集成的路径。
• 促进新应用领域中算法的快速原型设计与专门化。

提出的方法

• 描述一种模块化的面向对象软件架构，使用 Problem/ProblemLite 类来定义问题，使用 Optimizer 基类来实现算法。
• 实现用于海森近似、直线搜索和准则函数的模块，可以被优化器组合使用。
• 提供对外部优化器（例如 SNOPT、IPOPT）以及建模语言（OpenMDAO、CSDL、Jax、CasADi）的接口。
• 包括用于非面向对象用法的过程化接口（optimize）以及内置的可视化/记录工具。
• 在解析导数不可用时，支持数值微分，包括 JVPs、VJPs 和 HVPs，并支持缩放和热启动。

Figure 2 : UML class diagram for modOpt

实验结果

研究问题

• RQ1 如何通过模块化组件构建优化算法，以实现快速原型设计和专门化？
• RQ2 在给定问题上，统一接口对切换模型和优化器的影响是什么？
• RQ3 modOpt 如何与多种建模语言以及遗留优化器包集成？
• RQ4 内置教学算法和可视化工具带来哪些教育与基准测试方面的收益？
• RQ5 modOpt 能否在可扩展的导数处理下支持广义问题类（QP、凸优化、NLP）？

主要发现

• modOpt 提供了统一的、模块化的体系结构，将 Problem/ProblemLite 模型与 Optimizer 实现耦合在一起。
• 它包含用于直线搜索、海森近似和多种准则函数的模块，便于灵活的算法构造。
• 该框架提供 OpenMDAO、CSDL、Jax、CasADi 的接口，以及对外部优化器如 SNOPT 和 SLSQP 的接口。
• 过程化的 optimize 接口以及内置的可视化/记录支持有助于学习、测试和基准测试。
• 教学算法和透明的 Python 实现支持教育和快速实验。
• 基于 HDF5 的记录与热启动功能提升可重复性与迭代改进。

Figure 3 : Convergence comparison of different algorithms using the quartic function. Every algorithm starts from the initial guess $(1,1)$ . The total number of objective function evaluations required for convergence is indicated in parentheses next to the name of each algorithm in the legend.