[Paper Review] Adaptive estimation of spectral densities via wavelet thresholding and information projection

Jérémie Bigot, Rolando Biscay Lirio+2 more|arXiv (Cornell University)|Dec 10, 2009

Image and Signal Denoising Methods20 references3 citations

TL;DR

This paper proposes a wavelet-based adaptive estimator for spectral densities of stationary Gaussian processes by combining wavelet thresholding with information projection onto exponential families, ensuring non-negativity. The method achieves optimal Kullback-Leibler discrepancy rates over Besov classes, outperforming traditional linear methods in inhomogeneous smoothness scenarios.

ABSTRACT

In this paper, we study the problem of adaptive estimation of the spectral density of a stationary Gaussian process. For this purpose, we consider a wavelet-based method which combines the ideas of wavelet approximation and estimation by information projection in order to warrants that the solution is a nonnegative function. The spectral density of the process is estimated by projecting the wavelet thresholding expansion of the periodogram onto a family of exponential functions. This ensures that the spectral density estimator is a strictly positive function. Then, by Bochner's theorem, the corresponding estimator of the covariance function is semidefinite positive. The theoretical behavior of the estimator is established in terms of rate of convergence of the Kullback-Leibler discrepancy over Besov classes. We also show the excellent practical performance of the estimator in some numerical experiments.

Motivation & Objective

• To develop a non-negative, adaptive estimator for spectral densities of stationary Gaussian processes that preserves non-negative definiteness of the covariance function.
• To overcome limitations of linear smoothing and standard wavelet thresholding, which may produce negative spectral density estimates.
• To establish theoretical optimality in terms of Kullback-Leibler discrepancy rather than $L_2$-risk, which is more natural for density estimation.
• To propose a computationally simple thresholding rule that adapts to inhomogeneous smoothness in spectral densities.
• To demonstrate superior finite-sample performance through numerical experiments compared to existing methods.

Proposed method

• Construct a nonlinear wavelet approximation of the periodogram using hard thresholding to reduce noise.
• Project the thresholded wavelet coefficients onto a family of exponential functions to ensure the resulting spectral density estimator is strictly positive.
• Use information projection to estimate the spectral density as the minimizer of Kullback-Leibler divergence within an exponential family model.
• Define the estimator as $ f_{j_1(n), heta_n^{ ext{HT}}}^{HT} $, where $ \theta_n $ is chosen to match the thresholded wavelet coefficients.
• Employ a thresholding rule based on local $ l_1 $-norms of the periodogram, avoiding kernel pre-estimation of variance.
• Establish theoretical bounds using Besov space norms and Kullback-Leibler discrepancy, with convergence rates derived via concentration inequalities and entropy considerations.

Experimental results

Research questions

• RQ1 Can wavelet thresholding combined with information projection yield a non-negative spectral density estimator that adapts to inhomogeneous smoothness?
• RQ2 What is the rate of convergence of the Kullback-Leibler discrepancy for this estimator over Besov classes?
• RQ3 How does the proposed method compare to existing linear and nonlinear estimators in terms of theoretical optimality and finite-sample performance?
• RQ4 Can the thresholding rule be simplified without sacrificing adaptivity or convergence rate?
• RQ5 Does the information projection step ensure that the estimator remains strictly positive while maintaining optimal estimation accuracy?

Key findings

• The proposed estimator achieves a Kullback-Leibler discrepancy rate of order $ \left(\frac{n}{\log n}\right)^{-\frac{2s}{2s+1}} $ over Besov classes $ F_{p,q}^s(M) $ with $ s > \frac{1}{p} $, matching the optimal rate for this loss function.
• The method ensures strict positivity of the spectral density estimator by construction, preserving the non-negative definiteness of the corresponding covariance function.
• Theoretical analysis shows that the estimation error converges to zero uniformly over $ F_{p,q}^s(M) $, with $ \epsilon_{j_1(n)} \to 0 $ as $ n \to \infty $, ensuring consistency.
• The thresholding rule is simple and avoids kernel pre-estimation of variance, making it robust to low-regularity spectral densities.
• Numerical experiments demonstrate excellent practical performance, outperforming existing methods in terms of estimation accuracy and stability.
• The estimator achieves optimal rates even when the spectral density exhibits inhomogeneous smoothness, where linear methods fail to achieve optimal $ L_2 $-rates.