[Paper Review] A Multi-Fidelity Active Learning Method for Global Design Optimization Problems with Noisy Evaluations

Riccardo Pellegrini, Jeroen Wackers+4 more|arXiv (Cornell University)|Feb 14, 2022

Advanced Multi-Objective Optimization Algorithms4 citations

TL;DR

This paper proposes a multi-fidelity active learning method using stochastic radial basis functions with adaptive hyperparameter optimization to improve global design optimization under noisy evaluations. By combining low-fidelity simulations with corrective error surrogates and selecting both design points and fidelity levels via lower confidence bounding, the method achieves superior accuracy and efficiency compared to high-fidelity-only approaches, especially in high-dimensional problems and noisy environments.

ABSTRACT

A multi-fidelity (MF) active learning method is presented for design optimization problems characterized by noisy evaluations of the performance metrics. Namely, a generalized MF surrogate model is used for design-space exploration, exploiting an arbitrary number of hierarchical fidelity levels, i.e., performance evaluations coming from different models, solvers, or discretizations, characterized by different accuracy. The method is intended to accurately predict the design performance while reducing the computational effort required by simulation-driven design (SDD) to achieve the global optimum. The overall MF prediction is evaluated as a low-fidelity trained surrogate corrected with the surrogates of the errors between consecutive fidelity levels. Surrogates are based on stochastic radial basis functions (SRBF) with least squares regression and in-the-loop optimization of hyperparameters to deal with noisy training data. The method adaptively queries new training data, selecting both the design points and the required fidelity level via an active learning approach. This is based on the lower confidence bounding method, which combines performance prediction and associated uncertainty to select the most promising design regions. The fidelity levels are selected considering the benefit-cost ratio associated with their use in the training. The method's performance is assessed and discussed using four analytical tests and three SDD problems based on computational fluid dynamics simulations, namely the shape optimization of a NACA hydrofoil, the DTMB 5415 destroyer, and a roll-on/roll-off passenger ferry. Fidelity levels are provided by both adaptive grid refinement and multi-grid resolution approaches. Under the assumption of a limited budget of function evaluations, the proposed MF method shows better performance in comparison with the model trained by high-fidelity evaluations only.

Motivation & Objective

• Address the challenge of noisy, computationally expensive simulations in simulation-driven design (SDD) for engineering optimization.
• Reduce computational cost while maintaining high accuracy in global optimization by leveraging multiple fidelity levels.
• Overcome the limitations of standard active learning in noisy environments, where models may overfit to noisy regions.
• Develop a dynamic, adaptive framework that selects both optimal design points and fidelity levels based on benefit-cost trade-offs.
• Improve exploration and exploitation balance in high-dimensional design spaces using multi-fidelity surrogate modeling with uncertainty-aware active learning.

Proposed method

• Construct a generalized multi-fidelity surrogate model using stochastic radial basis functions (SRBF) with least squares regression and in-the-loop hyperparameter optimization to handle noisy training data.
• Model the overall prediction as a low-fidelity surrogate corrected by error surrogates between consecutive fidelity levels, enabling hierarchical accuracy improvement.
• Implement an active learning strategy based on the lower confidence bound (LCB) criterion, which balances prediction and uncertainty to select the most informative design points.
• Dynamically select the fidelity level for each new evaluation based on a benefit-cost ratio, prioritizing low-cost, informative evaluations to reduce overall computational burden.
• Use adaptive grid refinement and multi-grid resolution as natural sources of fidelity levels, enabling consistent multi-fidelity data generation across CFD simulations.
• Optimize the number of RBF centers using clustering metrics (e.g., within-cluster sum of squares, silhouette) as an alternative to leave-one-out cross-validation.

Experimental results

Research questions

• RQ1 Can a multi-fidelity active learning framework outperform high-fidelity-only surrogate models in terms of accuracy and computational efficiency under noisy evaluations?
• RQ2 How does increasing the number of fidelity levels affect the performance and robustness of the surrogate model in high-dimensional design spaces?
• RQ3 To what extent does the LCB-based active learning strategy mitigate noise-induced overfitting and improve design space exploration?
• RQ4 How do adaptive fidelity selection and error correction mechanisms enhance model accuracy while minimizing expensive high-fidelity evaluations?
• RQ5 Can multi-grid and adaptive grid refinement methods serve as effective, naturally occurring fidelity levels in multi-fidelity optimization frameworks?

Key findings

• The multi-fidelity surrogate model with three fidelity levels consistently outperformed both two-fidelity and high-fidelity-only models in terms of prediction error, especially in high-dimensional problems (D=5 and D=10).
• For the NACA hydrofoil and DTMB 5415 destroyer problems, the three-fidelity MF model achieved lower error values than the two-fidelity model, demonstrating improved robustness and noise filtering.
• The active learning process enabled wider exploration of the design space due to higher uncertainty in low-fidelity predictions, particularly in noisy regions.
• The LCB-based selection strategy led to clustering of training points in noisy regions due to non-vanishing uncertainty at training points, which improved noise identification but reduced uniformity of exploration.
• The method achieved better performance than high-fidelity-only models under a limited function evaluation budget, particularly in high-dimensional and noisy environments.
• Adaptive grid refinement and multi-grid resolution were shown to be natural and effective sources of fidelity levels, fitting seamlessly into the multi-fidelity framework.