[Paper Review] Polynomial-Chaos-based Kriging

R. Schoebi, Bruno Sudret+1 more|arXiv (Cornell University)|Feb 13, 2015

Probabilistic and Robust Engineering Design64 references3 citations

TL;DR

This paper proposes PC-Kriging, a novel non-intrusive surrogate modeling technique that combines Polynomial Chaos Expansions (PCE) for global approximation with Kriging for local refinement. By using an adaptive least-angle regression algorithm to select a sparse set of orthonormal polynomials and integrating them into a universal Kriging framework, PC-Kriging achieves superior accuracy—especially with limited experimental designs—outperforming standalone PCE or Kriging on benchmark functions.

ABSTRACT

Computer simulation has become the standard tool in many engineering fields for designing and optimizing systems, as well as for assessing their reliability. To cope with demanding analysis such as optimization and reliability, surrogate models (a.k.a meta-models) have been increasingly investigated in the last decade. Polynomial Chaos Expansions (PCE) and Kriging are two popular non-intrusive meta-modelling techniques. PCE surrogates the computational model with a series of orthonormal polynomials in the input variables where polynomials are chosen in coherency with the probability distributions of those input variables. On the other hand, Kriging assumes that the computer model behaves as a realization of a Gaussian random process whose parameters are estimated from the available computer runs, i.e. input vectors and response values. These two techniques have been developed more or less in parallel so far with little interaction between the researchers in the two fields. In this paper, PC-Kriging is derived as a new non-intrusive meta-modeling approach combining PCE and Kriging. A sparse set of orthonormal polynomials (PCE) approximates the global behavior of the computational model whereas Kriging manages the local variability of the model output. An adaptive algorithm similar to the least angle regression algorithm determines the optimal sparse set of polynomials. PC-Kriging is validated on various benchmark analytical functions which are easy to sample for reference results. From the numerical investigations it is concluded that PC-Kriging performs better than or at least as good as the two distinct meta-modeling techniques. A larger gain in accuracy is obtained when the experimental design has a limited size, which is an asset when dealing with demanding computational models.

Motivation & Objective

• Address the computational infeasibility of running high-fidelity models thousands or millions of times in uncertainty quantification and reliability analysis.
• Overcome the limitations of standalone Polynomial Chaos Expansions (PCE) and Kriging by combining their strengths in global approximation and local refinement.
• Develop a unified, non-intrusive meta-modeling approach that leverages both global polynomial trends and local Gaussian process correlation.
• Improve surrogate model accuracy, particularly for small experimental designs, by optimally selecting a sparse set of orthonormal polynomials via adaptive algorithms.
• Enable efficient reliability analysis and optimization by reducing model evaluation costs while maintaining high predictive accuracy.

Proposed method

• Formulate a hybrid meta-model by combining PCE as the trend component with Kriging as the stochastic correction, resulting in a universal Kriging model.
• Use least-angle regression (LAR) to iteratively select the optimal sparse set of orthonormal polynomials that best represent the global behavior of the computational model.
• Implement two variants: Sequential-PC-Kriging (SPC-Kriging), which adds polynomials sequentially, and Optimal-PC-Kriging (OPC-Kriging), which selects the best subset based on leave-one-out (LOO) error.
• Estimate hyperparameters of the Kriging correlation function using maximum likelihood estimation (MLE) at each step of the LAR-based selection process.
• Use leave-one-out (LOO) error as a proxy for generalization error to guide the selection of the optimal polynomial set, minimizing model complexity without sacrificing accuracy.
• Validate the method on analytical benchmark functions with known reference solutions using a large validation set to compute relative generalization error.

Experimental results

Research questions

• RQ1 Can combining PCE and Kriging into a single surrogate model improve accuracy compared to using either method alone, especially for small experimental designs?
• RQ2 How does the adaptive selection of a sparse set of orthonormal polynomials via least-angle regression affect the performance and complexity of the resulting meta-model?
• RQ3 Does the OPC-Kriging variant, which selects the optimal subset of polynomials based on LOO error, yield better accuracy and lower complexity than SPC-Kriging, which adds polynomials sequentially?
• RQ4 To what extent does the hybrid PC-Kriging model reduce the number of required model evaluations while maintaining high predictive accuracy in uncertainty quantification tasks?
• RQ5 How does the performance of PC-Kriging compare to ordinary Kriging and PCE in terms of generalization error across different benchmark functions and sample sizes?

Key findings

• PC-Kriging consistently outperforms or matches the performance of standalone PCE and Kriging across all benchmark functions tested, particularly for small experimental designs.
• OPC-Kriging achieves comparable or better accuracy than SPC-Kriging while reducing the number of polynomials in the trend component, thus lowering model complexity.
• For the Ishigami function with N=128 samples, using only 27 out of 56 polynomials (half the set) yielded a meta-model nearly as accurate as the full set, indicating significant complexity reduction without loss in accuracy.
• The leave-one-out (LOO) error closely tracks the true relative generalization error, validating its use as a reliable criterion for selecting the optimal polynomial set.
• The adaptive experimental design strategy—adding samples in regions of interest (e.g., near limit state surfaces in reliability analysis)—can further enhance accuracy where it matters most.
• The method is especially advantageous when computational models are expensive, as it achieves high accuracy with minimal model evaluations, making it suitable for reliability and optimization tasks.