[Paper Review] Spatial Mapping with Gaussian Processes and Nonstationary Fourier Features

Jean-François Ton, Seth Flaxman+2 more|arXiv (Cornell University)|Nov 15, 2017

Gaussian Processes and Bayesian Inference32 references3 citations

TL;DR

This paper proposes a novel framework for learning nonstationary kernel functions in Gaussian processes using Fourier feature representations and neural networks, enabling flexible, data-driven modeling of complex spatial and spatiotemporal dependencies. By learning frequency parameters end-to-end with deep learning regularization, the method improves generalization and interpretability without increasing computational complexity, outperforming stationary kernels on time series and land surface temperature mapping tasks.

ABSTRACT

The use of covariance kernels is ubiquitous in the field of spatial statistics. Kernels allow data to be mapped into high-dimensional feature spaces and can thus extend simple linear additive methods to nonlinear methods with higher order interactions. However, until recently, there has been a strong reliance on a limited class of stationary kernels such as the Matern or squared exponential, limiting the expressiveness of these modelling approaches. Recent machine learning research has focused on spectral representations to model arbitrary stationary kernels and introduced more general representations that include classes of nonstationary kernels. In this paper, we exploit the connections between Fourier feature representations, Gaussian processes and neural networks to generalise previous approaches and develop a simple and efficient framework to learn arbitrarily complex nonstationary kernel functions directly from the data, while taking care to avoid overfitting using state-of-the-art methods from deep learning. We highlight the very broad array of kernel classes that could be created within this framework. We apply this to a time series dataset and a remote sensing problem involving land surface temperature in Eastern Africa. We show that without increasing the computational or storage complexity, nonstationary kernels can be used to improve generalisation performance and provide more interpretable results.

Motivation & Objective

• To overcome the limitations of fixed, stationary kernels like Matérn and squared exponential in Gaussian process regression.
• To develop a scalable, data-driven method for learning complex, nonstationary kernel functions directly from spatiotemporal data.
• To maintain computational efficiency while enabling greater model flexibility and improved generalization performance.
• To integrate deep learning regularization techniques (e.g., Gaussian dropout) to prevent overfitting in nonstationary kernel learning.
• To demonstrate the method’s utility on real-world time series and remote sensing datasets, particularly land surface temperature in Eastern Africa.

Proposed method

• The method uses a Fourier feature expansion to represent the kernel function, mapping inputs into a high-dimensional feature space via random or learnable frequencies.
• It formulates the kernel learning problem as a neural network with trigonometric activation functions, where frequency parameters are learned end-to-end using gradient-based optimization.
• The framework leverages the connection between random Fourier features, Gaussian processes, and single-layer neural networks to maintain interpretability while enabling nonstationarity.
• Regularization via Gaussian dropout is applied to the Fourier feature layer to prevent overfitting, especially important in high-dimensional or noisy data regimes.
• The marginal likelihood is optimized during training, and the resulting kernel structure is interpretable and adaptable to local data patterns.
• The method preserves the O(nm²) time and O(nm) storage complexity of standard Fourier GP approximations, making it scalable to large datasets.

Experimental results

Research questions

• RQ1 Can nonstationary kernel functions be learned efficiently and effectively from data using Fourier feature representations?
• RQ2 Does learning nonstationary kernels improve generalization performance in spatial and spatiotemporal modeling compared to stationary kernels?
• RQ3 Can the proposed framework maintain computational efficiency while enabling greater model flexibility and interpretability?
• RQ4 How does the integration of deep learning regularization techniques affect overfitting in nonstationary kernel learning?
• RQ5 To what extent can the method adapt to local data structures, such as varying spatial correlation patterns in land surface temperature data?

Key findings

• The proposed method achieves improved generalization performance on both a time series dataset and a land surface temperature mapping task in Eastern Africa compared to stationary kernel baselines.
• Nonstationary kernels learned via the framework provide more interpretable and plausible predictions, particularly in regions with heterogeneous spatial correlation structures.
• The method maintains the same computational and storage complexity (O(nm²) time, O(nm) storage) as standard Fourier GP approximations, enabling scalability to large datasets.
• Histograms of learned frequency parameters show distinct, non-uniform distributions, indicating that the model captures meaningful, data-dependent spectral patterns rather than relying on fixed or random frequencies.
• The covariance matrix images reveal location-specific correlation structures under nonstationarity, whereas stationary models produce identical patterns across all locations.
• The use of Gaussian dropout significantly reduces overfitting, as evidenced by stable marginal likelihood optimization and robust performance across validation sets.