[Paper Review] A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty

Sayan Ghosh, Jesper Kristensen+3 more|arXiv (Cornell University)|Jul 26, 2019

Advanced Multi-Objective Optimization Algorithms40 references4 citations

TL;DR

This paper proposes a novel adaptive sampling strategy for multi-fidelity Gaussian processes that jointly selects the next design point and fidelity level to maximize uncertainty reduction per unit cost. By introducing a fidelity-aware uncertainty-to-cost ratio criterion and integrating the 'Believer' concept to quantify predictive uncertainty reduction, the method achieves significantly lower total cost—especially at high fidelity cost ratios—while maintaining or improving prediction accuracy compared to baseline approaches.

ABSTRACT

Multi-fidelity Gaussian process is a common approach to address the extensive computationally demanding algorithms such as optimization, calibration and uncertainty quantification. Adaptive sampling for multi-fidelity Gaussian process is a changing task due to the fact that not only we seek to estimate the next sampling location of the design variable, but also the level of the simulator fidelity. This issue is often addressed by including the cost of the simulator as an another factor in the searching criterion in conjunction with the uncertainty reduction metric. In this work, we extent the traditional design of experiment framework for the multi-fidelity Gaussian process by partitioning the prediction uncertainty based on the fidelity level and the associated cost of execution. In addition, we utilize the concept of Believer which quantifies the effect of adding an exploratory design point on the Gaussian process uncertainty prediction. We demonstrated our framework using academic examples as well as a industrial application of steady-state thermodynamic operation point of a fluidized bed process

Motivation & Objective

• Address the challenge of efficiently allocating limited computational and experimental budgets across multiple-fidelity models in surrogate modeling.
• Overcome the limitations of traditional adaptive sampling that treat fidelity selection and design point selection as separate decisions.
• Develop a unified criterion that balances predictive uncertainty reduction with fidelity-specific costs to improve sampling efficiency.
• Incorporate the 'Believer' framework to quantify the impact of new data points on uncertainty reduction, enhancing decision-making in multi-fidelity settings.
• Demonstrate the method’s effectiveness across analytical test cases and a real industrial application in fluidized bed thermodynamic modeling.

Proposed method

• Propose a new adaptive sampling criterion, Maximum Individual Fidelity Uncertainty-to-Cost Ratio (Max IF-UCR), which evaluates the uncertainty reduction per unit cost for each fidelity level at each candidate design point.
• Partition the total predictive uncertainty into components attributable to low-fidelity and discrepancy (correction) models, enabling fidelity-specific uncertainty quantification.
• Integrate the 'Believer' concept from Gaussian process literature to assess the expected reduction in predictive uncertainty from adding a new data point, improving the informativeness of sampling decisions.
• Extend Max IF-UCR to Max IF-UCR bel by combining the fidelity-specific uncertainty reduction with the Believer metric for enhanced predictive gain estimation.
• Use a multi-fidelity Gaussian process framework where high-fidelity outputs are modeled as a low-fidelity prediction corrected by a discrepancy function.
• Apply the proposed criteria iteratively: select the next sampling point and fidelity with the highest IF-UCR or IF-UCR bel value, then update the surrogate model with new data.

Experimental results

Research questions

• RQ1 How can adaptive sampling in multi-fidelity Gaussian processes be improved by jointly optimizing the selection of design points and fidelity levels?
• RQ2 What is the impact of incorporating fidelity-specific uncertainty decomposition on sampling efficiency and cost reduction?
• RQ3 How does the 'Believer' metric enhance the selection of informative data points in multi-fidelity settings compared to global uncertainty metrics?
• RQ4 How does the performance of the proposed Max IF-UCR and Max IF-UCR bel criteria compare to the baseline Max MF-UCR criterion across varying fidelity cost ratios?
• RQ5 To what extent does the proposed method reduce total cost while maintaining or improving prediction accuracy in real-world engineering applications?

Key findings

• For a high-fidelity to low-fidelity cost ratio of 2:1, the proposed Max IF-UCR and Max IF-UCR bel methods showed minimal cost advantage over the baseline Max MF-UCR, as the cost difference was not significant.
• At higher cost ratios (5:1 and 10:1), both Max IF-UCR and Max IF-UCR bel achieved significantly lower total costs compared to the baseline Max MF-UCR, with Max IF-UCR yielding the best results.
• The Max IF-UCR bel variant converged to similar performance trends as Max IF-UCR at high cost ratios, indicating that the Believer extension improves robustness without sacrificing efficiency.
• Root mean squared error (RMSE) converged to similar ranges across all methods in the fluidized bed process case, indicating that prediction accuracy was maintained despite cost differences.
• The method demonstrated robustness across 10 independent runs with randomly chosen initial samples, confirming consistent performance under varying starting conditions.
• The adaptive sampling framework successfully reduced predictive uncertainty with fewer high-fidelity evaluations, particularly when high-fidelity simulations were expensive relative to low-fidelity ones.