[Paper Review] Proper Complex Gaussian Processes for Regression

Rafael Boloix-Tortosa, F. Javier Payán-Somet+2 more|arXiv (Cornell University)|Feb 17, 2015

Gaussian Processes and Bayesian Inference27 references3 citations

TL;DR

This paper proposes a novel framework for proper complex Gaussian processes in regression, leveraging a complex-valued kernel derived via convolutional cross-covariance to ensure isotropy and stationarity. By modeling similarity through the magnitude of complex input differences and using Wirtinger derivatives for hyperparameter learning, the method achieves improved estimation accuracy over prior approaches, particularly in non-canceling cross-covariance scenarios.

ABSTRACT

Complex-valued signals are used in the modeling of many systems in engineering and science, hence being of fundamental interest. Often, random complex-valued signals are considered to be proper. A proper complex random variable or process is uncorrelated with its complex conjugate. This assumption is a good model of the underlying physics in many problems, and simplifies the computations. While linear processing and neural networks have been widely studied for these signals, the development of complex-valued nonlinear kernel approaches remains an open problem. In this paper we propose Gaussian processes for regression as a framework to develop 1) a solution for proper complex-valued kernel regression and 2) the design of the reproducing kernel for complex-valued inputs, using the convolutional approach for cross-covariances. In this design we pay attention to preserve, in the complex domain, the measure of similarity between near inputs. The hyperparameters of the kernel are learned maximizing the marginal likelihood using Wirtinger derivatives. Besides, the approach is connected to the multiple output learning scenario. In the experiments included, we first solve a proper complex Gaussian process where the cross-covariance does not cancel, a challenging scenario when dealing with proper complex signals. Then we successfully use these novel results to solve some problems previously proposed in the literature as benchmarks, reporting a remarkable improvement in the estimation error.

Motivation & Objective

• To develop a principled framework for nonlinear regression using proper complex Gaussian processes, addressing limitations in existing complex kernel methods.
• To design a complex-valued reproducing kernel that preserves physical similarity measures in the complex domain, ensuring isotropy and stationarity.
• To enable hyperparameter learning via marginal likelihood maximization using Wirtinger derivatives, avoiding cross-validation.
• To extend the framework to multiple-output learning scenarios while maintaining properness and positive semi-definiteness of the kernel.
• To demonstrate superior performance on benchmark problems involving non-canceling cross-covariances, where prior methods fail.

Proposed method

• The method constructs a complex-valued kernel by combining real and imaginary parts of the covariance matrix, where the real part captures intra-part covariances and the imaginary part models cross-covariance between real and imaginary components.
• The kernel is designed using a convolutional approach on four basis functions, with parameterized exponential forms to ensure smoothness and physical interpretability.
• The cross-covariance structure is enforced to be skew-symmetric and positive semi-definite, ensuring the full covariance matrix remains a valid reproducing kernel.
• Similarity between complex inputs is measured via the absolute value of their complex difference, enabling isotropy and stationarity in the kernel design.
• Hyperparameters are optimized by maximizing the marginal likelihood using Wirtinger derivatives, a technique adapted for complex-valued functions.
• The framework is validated in both single- and multiple-output regression settings, with performance compared against existing benchmarks.

Experimental results

Research questions

• RQ1 How can a proper complex-valued kernel be designed to ensure isotropy and stationarity while modeling cross-covariance between real and imaginary parts?
• RQ2 Can a complex Gaussian process framework be constructed to maintain properness while enabling full Bayesian inference with probabilistic outputs?
• RQ3 How does the use of Wirtinger derivatives for marginal likelihood maximization improve hyperparameter learning in complex-valued regression?
• RQ4 What is the performance gain of the proposed kernel over existing complex kernels in scenarios with non-canceling cross-covariances?
• RQ5 To what extent can the proposed framework generalize to multiple-output learning tasks in complex-valued signal processing?

Key findings

• The proposed kernel achieves isotropy and stationarity by measuring similarity through the magnitude of the complex difference between inputs, enabling better modeling of physical systems.
• The method successfully handles challenging scenarios where cross-covariance does not cancel, a limitation in prior approaches.
• The framework reports a remarkable improvement in estimation error on benchmark problems previously used in the literature, demonstrating superior generalization.
• The kernel structure is validated to be positive semi-definite by construction, ensuring its validity as a reproducing kernel in a complex-valued RKHS.
• The use of Wirtinger derivatives enables efficient and accurate hyperparameter learning, avoiding the need for cross-validation.
• The design allows independent tuning of real and imaginary parts of the kernel, increasing flexibility without sacrificing properness or stability.