[Paper Review] The De-Biased Whittle Likelihood for Second-Order Stationary Stochastic Processes

Adam M. Sykulski, Sofia C. Olhede+1 more|arXiv (Cornell University)|May 22, 2016

Statistical Methods and Inference31 references5 citations

TL;DR

This paper proposes a de-biased Whittle likelihood method for second-order stationary stochastic processes that corrects bias in parameter estimates while maintaining the standard $Ó(n\log n)$ computational efficiency. The approach ensures consistency and significantly improves estimation accuracy, especially when combined with existing bias reduction techniques like tapering and differencing.

ABSTRACT

The Whittle likelihood is a computationally efficient pseudo-maximum likelihood inference procedure which is known to produce biased parameter estimates for large classes of time series models. We propose a method for de-biasing Whittle likelihood parameter estimates for second-order stationary stochastic processes. We demonstrate how to compute the de-biased Whittle likelihood in the same $\mathcal{O}(n\log n)$ computational efficiency as standard Whittle likelihood. We prove that the method is consistent, and demonstrate its superior performance in simulation studies. We also demonstrate how the method can be easily combined with standard methods of bias reduction, such as tapering and differencing, to further reduce bias in parameter estimates.

Motivation & Objective

• To address the well-known issue of bias in Whittle likelihood parameter estimates for second-order stationary stochastic processes.
• To develop a computationally efficient method that corrects this bias without sacrificing the speed of standard Whittle likelihood.
• To ensure the de-biased method remains consistent and applicable to a broad class of time series models.
• To demonstrate compatibility with established bias reduction techniques such as tapering and differencing.
• To validate the method through rigorous simulation studies and theoretical consistency proofs.

Proposed method

• The method introduces a correction term to the standard Whittle likelihood to counteract known bias in parameter estimates for second-order stationary processes.
• The de-biasing correction is derived using asymptotic expansions of the likelihood, enabling analytical computation of bias adjustments.
• The corrected likelihood maintains the same $Ó(n\log n)$ computational complexity as the original Whittle likelihood by leveraging the fast Fourier transform (FFT).
• The approach is formulated in the spectral domain, allowing efficient computation via spectral density estimation.
• The method is designed to be modular, enabling seamless integration with existing bias mitigation strategies like tapering and differencing.
• Consistency of the de-biased estimator is formally proven under regularity conditions for second-order stationary processes.

Experimental results

Research questions

• RQ1 Can the Whittle likelihood be systematically de-biased while preserving its computational efficiency?
• RQ2 How does the de-biased Whittle likelihood perform in comparison to standard Whittle likelihood in terms of bias and mean squared error?
• RQ3 To what extent can the de-biased method be combined with existing bias reduction techniques like tapering and differencing?
• RQ4 Is the de-biased Whittle estimator consistent under standard regularity conditions for second-order stationary processes?
• RQ5 What is the computational overhead of the de-biasing procedure compared to the original Whittle likelihood?

Key findings

• The de-biased Whittle likelihood achieves the same $Ó(n\log n)$ computational complexity as the standard Whittle likelihood, ensuring scalability for large datasets.
• The method is proven to be consistent, meaning parameter estimates converge in probability to the true values as sample size increases.
• Simulation studies show that the de-biased estimator significantly reduces bias compared to the standard Whittle likelihood across various time series models.
• The method demonstrates improved mean squared error performance, particularly in small to moderate sample sizes where bias is most pronounced.
• The de-biased likelihood can be effectively combined with tapering and differencing, leading to further reductions in estimation bias.
• The theoretical framework allows for analytical derivation of bias corrections, avoiding the need for computationally intensive resampling methods.