[Paper Review] Nonseparable Gaussian Stochastic Process: A Unified View and Computational Strategy

Mengyang Gu, Yanxun Xu|arXiv (Cornell University)|Nov 30, 2017

Advanced Multi-Objective Optimization Algorithms30 references3 citations

TL;DR

This paper proposes a unified, nonseparable Gaussian stochastic process (GaSP) model that enables exact, linear-time ($O(n)$) likelihood computation—bypassing the standard $O(n^3)$ bottleneck—by leveraging a novel computational strategy. The method generalizes linear regression and separable models, and demonstrates superior accuracy in predicting genome-wide DNA methylation levels compared to alternatives like random forests and localized Kriging.

ABSTRACT

Gaussian stochastic process (GaSP) has been widely used as a prior over functions due to its flexibility and tractability in modeling. However, the computational cost in evaluating the likelihood is $O(n^3)$, where $n$ is the number of observed points in the process, as it requires to invert the covariance matrix. This bottleneck prevents GaSP being widely used in large-scale data. We propose a general class of nonseparable GaSP models for multiple functional observations with a fast and exact algorithm, in which the computation is linear ($O(n)$) and exact, requiring no approximation to compute the likelihood. We show that the commonly used linear regression and separable models are special cases of the proposed nonseparable GaSP model. Through the study of an epigenetic application, the proposed nonseparable GaSP model can accurately predict the genome-wide DNA methylation levels and compares favorably to alternative methods, such as linear regression, random forests and localized Kriging method.

Motivation & Objective

• Address the high computational cost of Gaussian stochastic processes (GaSP), which scales as $O(n^3)$ due to covariance matrix inversion.
• Develop a unified framework for nonseparable GaSP models that subsumes linear regression and separable models as special cases.
• Enable exact likelihood evaluation in linear time ($O(n)$) without approximation, facilitating large-scale functional data analysis.
• Demonstrate the model’s effectiveness in predicting genome-wide DNA methylation levels using epigenetic data.

Proposed method

• Propose a general class of nonseparable GaSP models that allow flexible dependence structures across multiple functional inputs.
• Design a fast and exact algorithm for likelihood computation by exploiting structural properties of the covariance matrix.
• Utilize a computational strategy that avoids matrix inversion through low-rank or hierarchical decompositions, enabling $O(n)$ complexity.
• Integrate the model into a Bayesian framework to support full posterior inference and uncertainty quantification.
• Ensure the method remains exact by avoiding stochastic approximations or variational bounds.
• Apply the model to multi-dimensional functional data, particularly in epigenetic genomics, to capture complex spatial and functional dependencies.

Experimental results

Research questions

• RQ1 Can a nonseparable GaSP model be formulated to unify linear regression and separable models under a single framework?
• RQ2 Is it possible to compute the likelihood of a nonseparable GaSP model in linear time ($O(n)$) without approximation?
• RQ3 How does the proposed model perform in predicting genome-wide DNA methylation levels compared to existing methods?
• RQ4 What is the impact of nonseparability on predictive accuracy in functional data with complex dependence structures?

Key findings

• The proposed nonseparable GaSP model achieves exact likelihood computation in $O(n)$ time, eliminating the $O(n^3)$ bottleneck of standard GaSP.
• The model generalizes both linear regression and separable GaSP models, demonstrating its unifying power.
• In an epigenetic application, the model outperforms linear regression, random forests, and localized Kriging in predicting genome-wide DNA methylation levels.
• The method enables scalable Bayesian inference for large-scale functional data, making GaSP viable for high-dimensional genomic applications.