[Paper Review] Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes

Ali Hebbal, Loïc Brevault+3 more|arXiv (Cornell University)|Jun 29, 2020

Probabilistic and Robust Engineering Design57 references47 citations

TL;DR

The paper extends multi-fidelity deep Gaussian processes (MF-DGP) to handle differing input spaces across fidelities by embedding a non-parametric Bayesian input mapping within MF-DGP (MF-DGP-EM), enabling joint optimization and uncertainty quantification. It demonstrates improved prediction accuracy and uncertainty handling on analytical and physical problems compared to standard MF-DGP and space-mapping approaches.

ABSTRACT

Multi-fidelity approaches combine different models built on a scarce but accurate data-set (high-fidelity data-set), and a large but approximate one (low-fidelity data-set) in order to improve the prediction accuracy. Gaussian Processes (GPs) are one of the popular approaches to exhibit the correlations between these different fidelity levels. Deep Gaussian Processes (DGPs) that are functional compositions of GPs have also been adapted to multi-fidelity using the Multi-Fidelity Deep Gaussian process model (MF-DGP). This model increases the expressive power compared to GPs by considering non-linear correlations between fidelities within a Bayesian framework. However, these multi-fidelity methods consider only the case where the inputs of the different fidelity models are defined over the same domain of definition (e.g., same variables, same dimensions). However, due to simplification in the modeling of the low-fidelity, some variables may be omitted or a different parametrization may be used compared to the high-fidelity model. In this paper, Deep Gaussian Processes for multi-fidelity (MF-DGP) are extended to the case where a different parametrization is used for each fidelity. The performance of the proposed multifidelity modeling technique is assessed on analytical test cases and on structural and aerodynamic real physical problems.

Motivation & Objective

• Motivate and address the limitation that HF and LF models can have different input parametrizations or dimensions.
• Develop MF-DGP-EM to jointly learn input-space mappings and multi-fidelity relations in a non-parametric Bayesian framework.
• Preserve input-space correlations while enabling cross-fidelity information transfer.
• Evaluate the approach on analytical and real physical (structural and aerodynamic) problems and compare to existing methods.

Proposed method

• Formulate MF-DGP-EM as a two-level deep Gaussian process where each layer corresponds to a fidelity, with a mapping GP between successive input spaces.
• Introduce multi-output input-mapping GPs conditioned on the nominal mapped HF data to transform HF inputs to LF inputs.
• Incorporate the input mapping into the MF-DGP covariance via kl(x(i), x(j)) = kρ_l(x(i), x(j)) × kf−1_l(f∗−1_l(x(i)), f∗−1_l(x(j))) + kγ_l(x(i), x(j)).
• Use inducing inputs per layer and a variational (doubly stochastic) ELBO objective to train the model.
• Adopt natural-gradient steps for variational parameters and stochastic gradient steps for hyperparameters; constrain inducing inputs to maintain dependency on previous layer predictions as in (13).
• Predict at new points by propagating through the first level to obtain lower-fidelity projections, then through the second level to obtain multi-fidelity predictions.

Experimental results

Research questions

• RQ1 How can MF-DGP be extended to handle different input space parametrizations across fidelities?
• RQ2 Can a non-parametric, Bayesian input mapping embedded in MF-DGP improve predictive accuracy and uncertainty quantification when input spaces differ?
• RQ3 What is the impact of jointly learning the input mapping and MF-DGP on predictive performance compared to fixed mappings or standard MF-DGP?
• RQ4 How does MF-DGP-EM perform on analytical and real-world structural/aerodynamic multi-fidelity problems relative to MF-DGP and space-mapping baselines?

Key findings

• MF-DGP-EM improves prediction and uncertainty quantification over MF-DGP when fidelities have different input spaces (e.g., RMSE 0.077 vs 0.255; MNLL −1.921 vs 0.231 on illustrative comparisons).
• The MF-DGP-EM approach yields robust performance across varying HF data set sizes and outperforms bias-correction and GP-HF baselines in multiple setups.
• On Problem 1 with varying HF data sizes, MF-DGP-EM achieved better RMSE and MNLL than GP-HF and BC across distributions of HF data points, indicating robustness to DoE variation.
• In Problem 2, MF-DGP-EM generally improved RMSE and uncertainty quantification as HF data size increased, outperforming BC and GP-HF in most scenarios.
• The approach enables a concurrent optimization of the mapping and the MF-DGP model, leveraging a non-parametric mapping and maintaining correlations in the original input spaces.