[Paper Review] A Novel Approach to Forecasting Financial Volatility with Gaussian Process Envelopes

Syed Ali Asad Rizvi, Stephen Roberts+2 more|arXiv (Cornell University)|May 2, 2017

Gaussian Processes and Bayesian Inference12 references3 citations

TL;DR

This paper proposes a novel Gaussian Process (GP) regression approach that forecasts financial volatility by modeling the upper and lower envelopes of return time series, outperforming traditional GARCH models by 50% in mean squared error and 20% over a random walk on four currency pairs over a 2-year half-hourly period.

ABSTRACT

In this paper we use Gaussian Process (GP) regression to propose a novel approach for predicting volatility of financial returns by forecasting the envelopes of the time series. We provide a direct comparison of their performance to traditional approaches such as GARCH. We compare the forecasting power of three approaches: GP regression on the absolute and squared returns; regression on the envelope of the returns and the absolute returns; and regression on the envelope of the negative and positive returns separately. We use a maximum a posteriori estimate with a Gaussian prior to determine our hyperparameters. We also test the effect of hyperparameter updating at each forecasting step. We use our approaches to forecast out-of-sample volatility of four currency pairs over a 2 year period, at half-hourly intervals. From three kernels, we select the kernel giving the best performance for our data. We use two published accuracy measures and four statistical loss functions to evaluate the forecasting ability of GARCH vs GPs. In mean squared error the GP's perform 20% better than a random walk model, and 50% better than GARCH for the same data.

Motivation & Objective

• To address the limitations of parametric models like GARCH in capturing asymmetric volatility and non-linear dynamics in financial returns.
• To explore non-parametric Bayesian methods, specifically Gaussian Processes, for robust volatility forecasting without strong distributional assumptions.
• To evaluate whether modeling the envelope of returns—rather than raw returns—improves forecasting accuracy and captures volatility clustering and asymmetry.
• To compare GP-based approaches against GARCH and random walk benchmarks using rigorous statistical metrics and loss functions.
• To investigate the impact of hyperparameter updating and kernel selection on forecasting performance in a real-world financial setting.

Proposed method

• Model the upper and lower envelopes of absolute or squared returns using Gaussian Process regression in log-space to stabilize variance and improve smoothness.
• Apply GP regression to separate positive and negative return series, then average forecasts to capture asymmetric volatility effects.
• Use maximum a posteriori estimation with a Gaussian prior to infer hyperparameters, ensuring robustness and regularization.
• Transform GP predictions from log-space back to original scale using exponential functions to recover volatility forecasts and 95% credible intervals.
• Employ Cholesky updating and downdating to efficiently maintain the covariance matrix during sequential forecasting, enabling real-time performance.
• Select the optimal kernel from three candidates (e.g., squared exponential, Matérn) based on out-of-sample forecasting performance on the data.

Experimental results

Research questions

• RQ1 Can Gaussian Process regression on return envelopes outperform traditional GARCH models in forecasting financial volatility?
• RQ2 How does modeling positive and negative returns separately improve volatility prediction compared to treating returns as symmetric?
• RQ3 What is the impact of hyperparameter updating at each forecasting step on prediction accuracy?
• RQ4 Which GP kernel structure yields the best out-of-sample forecasting performance for FX volatility?
• RQ5 How do GP-based forecasts compare to random walk and GARCH in terms of mean squared error and statistical loss functions?

Key findings

• The GP-based envelope model reduced mean squared error by 50% compared to GARCH and 20% compared to a random walk on four major currency pairs over a 2-year half-hourly period.
• Modeling the envelope of absolute returns improved robustness to noise and captured volatility clustering more effectively than direct regression on returns.
• Separate modeling of positive and negative return envelopes significantly enhanced performance by capturing the asymmetric impact of market movements on volatility.
• The squared exponential kernel outperformed other kernels in terms of forecasting accuracy, as determined by loss function minimization.
• Hyperparameter updating at each forecasting step improved model adaptability and contributed to better overall performance.
• The GP approach demonstrated superior robustness to outliers and maintained stable predictive intervals, unlike GARCH which assumes conditional normality.