[論文レビュー] Multi-Fidelity Methods for Optimization: A Survey

Ke Li, Fan Li|arXiv (Cornell University)|Feb 15, 2024

Metaheuristic Optimization Algorithms Research被引用数 7

ひとこと要約

1-2文の直接的な要約: 多忠実度最適化（MFO）の総合的な調査で、新規のテキストマイニングフレームワークとサロゲートモデル、忠実度管理、および最適化アルゴリズムの体系的分析を組み合わせ、応用と今後の課題を概説する。

ABSTRACT

Real-world black-box optimization often involves time-consuming or costly experiments and simulations. Multi-fidelity optimization (MFO) stands out as a cost-effective strategy that balances high-fidelity accuracy with computational efficiency through a hierarchical fidelity approach. This survey presents a systematic exploration of MFO, underpinned by a novel text mining framework based on a pre-trained language model. We delve deep into the foundational principles and methodologies of MFO, focusing on three core components -- multi-fidelity surrogate models, fidelity management strategies, and optimization techniques. Additionally, this survey highlights the diverse applications of MFO across several key domains, including machine learning, engineering design optimization, and scientific discovery, showcasing the adaptability and effectiveness of MFO in tackling complex computational challenges. Furthermore, we also envision several emerging challenges and prospects in the MFO landscape, spanning scalability, the composition of lower fidelities, and the integration of human-in-the-loop approaches at the algorithmic level. We also address critical issues related to benchmarking and the advancement of open science within the MFO community. Overall, this survey aims to catalyze further research and foster collaborations in MFO, setting the stage for future innovations and breakthroughs in the field.

研究の動機と目的

• Provide a holistic overview of multi-fidelity optimization (MFO) and its key components.
• Introduce a text-mining framework to map the MFO literature and identify trends.
• Classify and analyze multi-fidelity surrogate models and fidelity management strategies.
• Summarize optimization techniques used in MFO, with emphasis on Bayesian optimization and surrogate-assisted methods.
• Discuss benchmarks, applications, challenges, and future directions in MFO.

提案手法

• Develop a closed-loop text-mining framework using pre-trained language models to curate and analyze 1,242 MFO articles from 1998–2023.
• Apply BERTopic (Sentence-BERT, UMAP, HDBSCAN, TF-IDF, KeyBERT) to extract 23 topics and build a taxonomy.
• Provide in-depth reviews of seven major multi-fidelity surrogate modeling methods and their formulations (single-model, space-mapping, correction-based, AR1, MTGP, nonlinear hierarchical, MF-PINNs).
• Survey optimizers for MFO with emphasis on Bayesian optimization and surrogate-assisted evolutionary algorithms, plus fidelity management strategies (fixed vs adaptive).
• Discuss benchmark development and three primary application domains: hyperparameter optimization, engineering design, and scientific discovery.

実験結果

リサーチクエスチョン

• RQ1 What are the core components and relationships that define multi-fidelity optimization?
• RQ2 How can literature be systematically mapped to reveal trends, venues, and geographic contributions in MFO?
• RQ3 What surrogate modeling families are most prevalent, and what are their strengths/limitations across fidelity relationships?
• RQ4 What optimization strategies and fidelity-management schemes are most effective in MFO?
• RQ5 What are the main applications, benchmarks, challenges, and future directions in MFO?

主な発見

• The authors identify seven major families of multi-fidelity surrogate models (single-model, space-mapping, correction-based, autoregressive AR1, multi-task Gaussian processes, nonlinear hierarchical, and multi-fidelity PINNs).
• Text mining of 1,242 papers (1998–2023) reveals rising MFO activity, with Structural and Multidisciplinary Optimization as a leading venue and USA/China/Europe as top regions.
• 23 distinct topics emerge, with ‘model and optimization’ as the most active theme, and three main application domains: engineering design, ML hyperparameter optimization, and scientific discovery.
• Bayesian optimization (BO) and surrogate-assisted evolutionary algorithms (SAEA) are central to MFBO, complemented by bandit optimization for hyperparameter tasks.
• No single surrogate method dominates across all problems (no free lunch), underscoring the need for problem-specific method selection.
• The survey discusses challenges in scalability, fidelity composition, human-in-the-loop integration, benchmarking, and open science in MFO.