[Paper Review] Extrinsic Gaussian processes for regression and classification on manifolds

Lizhen Lin, Mu Niu+2 more|arXiv (Cornell University)|Jun 27, 2017

Morphological variations and asymmetry25 references3 citations

TL;DR

This paper introduces extrinsic Gaussian processes (eGPs) for regression and classification on manifolds by embedding the manifold into a Euclidean space and constructing kernels on the embedded image. The method inherits favorable theoretical properties from Euclidean GPs and achieves posterior contraction at optimal rates for smooth functions on compact manifolds.

ABSTRACT

Gaussian processes (GPs) are very widely used for modeling of unknown functions or surfaces in applications ranging from regression to classification to spatial processes. Although there is an increasingly vast literature on applications, methods, theory and algorithms related to GPs, the overwhelming majority of this literature focuses on the case in which the input domain corresponds to a Euclidean space. However, particularly in recent years with the increasing collection of complex data, it is commonly the case that the input domain does not have such a simple form. For example, it is common for the inputs to be restricted to a non-Euclidean manifold, a case which forms the motivation for this article. In particular, we propose a general extrinsic framework for GP modeling on manifolds, which relies on embedding of the manifold into a Euclidean space and then constructing extrinsic kernels for GPs on their images. These extrinsic Gaussian processes (eGPs) are used as prior distributions for unknown functions in Bayesian inferences. Our approach is simple and general, and we show that the eGPs inherit fine theoretical properties from GP models in Euclidean spaces. We consider applications of our models to regression and classification problems with predictors lying in a large class of manifolds, including spheres, planar shape spaces, a space of positive definite matrices, and Grassmannians. Our models can be readily used by practitioners in biological sciences for various regression and classification problems, such as disease diagnosis or detection. Our work is also likely to have impact in spatial statistics when spatial locations are on the sphere or other geometric spaces.

Motivation & Objective

• Address the lack of practical and theoretically grounded GP models for regression and classification with manifold-valued predictors.
• Overcome the challenge of constructing valid covariance kernels for GPs on non-Euclidean manifolds.
• Develop a general, implementable framework that preserves key theoretical properties of standard GPs in Euclidean spaces.
• Enable accurate uncertainty quantification and prediction in applications involving complex data structures such as brain imaging, shape spaces, and positive definite matrices.
• Provide a scalable and flexible approach applicable to diverse manifolds including spheres, Grassmannians, and shape spaces.

Proposed method

• Embed a smooth d-dimensional manifold M into a higher-dimensional Euclidean space ℝ^D using an equivariant embedding J.
• Define extrinsic kernels on the image manifold J(M) ⊂ ℝ^D using standard Euclidean kernels, such as the squared exponential kernel.
• Construct an eGP prior on the original manifold M via pullback of the GP on J(M), ensuring the resulting process inherits smoothness and regularity.
• Use the eGP as a prior in Bayesian regression and classification, with likelihoods defined conditionally on the input manifold.
• Establish posterior contraction rates by leveraging the one-to-one correspondence between functions on M and their images on J(M), reducing the problem to standard GP theory in ℝ^D.
• Apply theoretical results from Yang and Dunson (2016) to derive posterior concentration rates under both fixed and random design settings.

Experimental results

Research questions

• RQ1 Can a general and practical framework be developed for Gaussian process regression and classification on arbitrary manifolds?
• RQ2 How can valid and computationally feasible covariance kernels be constructed for GPs on non-Euclidean manifolds?
• RQ3 To what extent do extrinsic GPs inherit the theoretical properties of standard GPs in Euclidean spaces?
• RQ4 What are the posterior contraction rates of eGPs for regression functions on smooth manifolds?
• RQ5 Can the framework be applied effectively to real-world problems such as neuroimaging and spatial statistics on geometric domains?

Key findings

• The eGP framework provides a general and implementable solution for GP modeling on manifolds by embedding into Euclidean space and constructing extrinsic kernels.
• eGPs inherit key theoretical properties of standard GPs, including posterior consistency and optimal contraction rates.
• For fixed design, the posterior contracts to the true regression function at rate εₙ = n^(-s/(2s+d)) (log n)^(d+1) for s-Hölder smooth functions with s ≤ 2.
• Under random design, the posterior contraction rate is achieved in L²(g) norm with respect to the design measure g, under the same smoothness conditions.
• The method is applicable to a wide range of manifolds, including spheres, shape spaces, positive definite matrices, and Grassmannians.
• The framework enables accurate uncertainty quantification and prediction in applications such as disease diagnosis using DTI data or landmark-based brain imaging.