[Paper Review] Fast multilevel sparse Gaussian kernels for high-dimensional approximation and integration

Zhaonan Dong, Emmanuil H. Georgoulis+2 more|arXiv (Cornell University)|Jan 14, 2015

Probabilistic and Robust Engineering Design13 references3 citations

TL;DR

This paper proposes a fast multilevel sparse kernel-based interpolation (MLSKI) method using directionally scaled tensor-product Gaussian kernels on structured sparse grids for high-dimensional approximation and integration in dimensions 5–10. By exploiting the tensor-product structure of anisotropic Gaussians, it enables fully parallelizable computation via precomputed one-dimensional cardinal basis functions, achieving high accuracy with reduced computational cost and demonstrating optimal convergence rates for both interpolation and numerical integration problems.

ABSTRACT

A fast multilevel algorithm based on directionally scaled tensor-product Gaussian kernels on structured sparse grids is proposed for interpolation of high-dimensional functions and for the numerical integration of high-dimensional integrals. The algorithm is based on the recent Multilevel Sparse Kernel-based Interpolation (MLSKI) method (Georgoulis, Levesley \& Subhan, \emph{SIAM J. Sci. Comput.}, 35(2), pp.~A815--A831, 2013), with particular focus on the fast implementation of Gaussian-based MLSKI for interpolation and integration problems of high-dimen-sional functions $f:[0,1]^d o\mathbb{R}$, with $5\le d\le 10$. The MLSKI interpolation procedure is shown to be interpolatory and a fast implementation is proposed. More specifically, exploiting the tensor-product nature of anisotropic Gaussian kernels, one-dimensional cardinal basis functions on a sequence of hierarchical equidistant nodes are precomputed to machine precision, rendering the interpolation problem into a fully parallelisable ensemble of linear combinations of function evaluations. A numerical integration algorithm is also proposed, based on interpolating the (high-dimensional) integrand. A series of numerical experiments highlights the applicability of the proposed algorithm for interpolation and integration for up to 10-dimensional problems.

Motivation & Objective

• To address the curse of dimensionality in high-dimensional function approximation and integration for $ d \geq 5 $.
• To develop a fast, scalable, and accurate algorithm for interpolation and numerical integration using sparse grids and Gaussian kernels.
• To ensure interpolatory properties and high accuracy in high-dimensional problems by exploiting tensor-product structure of anisotropic Gaussian kernels.
• To enable efficient computation through precomputation of one-dimensional cardinal functions and fully parallelizable linear combinations.
• To validate the method on benchmark problems including smooth, non-smooth, and high-dimensional integrands.

Proposed method

• The method uses a multilevel framework based on hierarchical sparse grids constructed via tensor-product Gaussian kernels with directionally scaled anisotropy.
• It decomposes the high-dimensional interpolation problem into independent one-dimensional subproblems using tensor-product cardinal basis functions precomputed to machine precision.
• Each level performs interpolation on a sparse grid by combining results from lower levels, ensuring interpolatory convergence.
• The integration scheme computes the integral by first interpolating the integrand using MLSKI and then numerically integrating the interpolant via precomputed quadrature weights.
• Quadrature weights are derived analytically from the integral of univariate cardinal functions, using the error function for high-precision evaluation.
• The algorithm is fully parallelizable, as each one-dimensional interpolation is independent and computed via linear combinations of function evaluations.

Experimental results

Research questions

• RQ1 Can a multilevel sparse kernel-based interpolation method using Gaussian kernels achieve high accuracy and fast computation in high-dimensional problems ($ d \leq 10 $)?
• RQ2 Does the tensor-product structure of anisotropic Gaussian kernels enable a fully parallelizable and efficient implementation of MLSKI?
• RQ3 Can the method maintain interpolatory properties and optimal convergence rates for both smooth and non-smooth functions in high dimensions?
• RQ4 How does the method perform in numerical integration tasks for high-dimensional integrands, especially those with boundary singularities or non-tensor product structure?
• RQ5 What is the convergence behavior and computational efficiency of the method across different benchmark functions in 5D and 10D?

Key findings

• For the 5D tensor product function $ f({\bf x}) = \prod_{i=1}^{5} 4x_i(1-x_i) $, the MLSKI method achieved a relative error of $ 5.24 \times 10^{-7} $ with 754,845 nodes.
• For the 10D function $ g({\bf x}) = \prod_{i=1}^{10} e^{-x_i(1-x_i)} $, the relative error dropped to $ 6.16 \times 10^{-7} $ with 331,777 nodes, demonstrating robust convergence.
• The method achieved a relative error of $ 3.23 \times 10^{-6} $ for the 4D Franke function with 331,777 nodes, showing effectiveness on non-tensor product, smooth functions.
• For the 5D payoff function with boundary derivative discontinuities, the relative error reached $ 9.24 \times 10^{-6} $ with 754,845 nodes, indicating good performance on non-smooth problems.
• The integration scheme achieved high accuracy by combining MLSKI interpolation with precomputed quadrature weights derived from the error function, enabling machine-precision evaluation of integrals.
• The algorithm's computational cost scales as $ N(\log N)^{d-1} $, avoiding the exponential cost of full grids while maintaining algebraic convergence rates independent of dimension.