[Paper Review] Multivariate Smoothing via the Fourier Integral Theorem and Fourier Kernel

Nhat Ho, Stephen G. Walker|arXiv (Cornell University)|Dec 28, 2020

Statistical Methods and Inference41 references4 citations

TL;DR

This paper introduces a novel multivariate smoothing method using the Fourier integral theorem and Fourier kernel, enabling nonparametric estimation of densities, regression functions, and mixing densities without requiring bandwidth covariance matrices. It achieves superior convergence rates compared to standard kernel methods by leveraging the product of one-dimensional Fourier kernels, which inherently preserve dependence structures across dimensions.

ABSTRACT

Starting with the Fourier integral theorem, we present natural Monte Carlo estimators of multivariate functions including densities, mixing densities, transition densities, regression functions, and the search for modes of multivariate density functions (modal regression). Rates of convergence are established and, in many cases, provide superior rates to current standard estimators such as those based on kernels, including kernel density estimators and kernel regression functions. Numerical illustrations are presented.

Motivation & Objective

• To develop a nonparametric multivariate smoothing framework based on the Fourier integral theorem that avoids the need for bandwidth covariance matrices.
• To address the limitations of traditional kernel methods—especially the complexity of bandwidth selection and covariance structure estimation in multivariate settings.
• To establish theoretical convergence rates for Fourier kernel estimators that outperform standard kernel-based methods in density and regression estimation.
• To extend the use of the Fourier kernel, previously underutilized in multivariate regression, to provide a natural product-kernel approach that preserves dependence structures.
• To provide a Monte Carlo-based estimation procedure for multivariate functions, including modal regression and transition densities, with rigorous theoretical guarantees.

Proposed method

• Utilizes the Fourier integral theorem to express a function $ m(x) $ as an integral involving $ \frac{\sin(R(y_j - x_j))}{y_j - x_j} $, enabling Monte Carlo approximation.
• Constructs the Fourier density estimator as $ \widehat{f}_{n,R}(x) = \frac{1}{n\pi^d} \sum_{i=1}^n \prod_{j=1}^d \frac{\sin(R(x_j - X_{ij}))}{x_j - X_{ij}} $, using i.i.d. samples.
• Extends the method to nonparametric regression via the Nadaraya–Watson-type estimator: $ \widehat{m}_{n,R}(x) = \frac{\sum_{i=1}^n Y_i \prod_{j=1}^d K_R(x_j - X_{ij})}{\sum_{i=1}^n \prod_{j=1}^d K_R(x_j - X_{ij})} $, where $ K_R(u) = \frac{\sin(Ru)}{u} $.
• Employs a product of one-dimensional Fourier kernels in higher dimensions, which automatically preserves dependence structures without requiring explicit covariance modeling.
• Applies empirical process theory and symmetrization techniques to bound the supremum of the empirical process, ensuring uniform convergence properties.
• Uses a multiplier bootstrap and concentration inequalities to establish asymptotic normality and convergence rates, with $ R^d $ scaling controlling bias-variance trade-off.

Experimental results

Research questions

• RQ1 Can the Fourier integral theorem be leveraged to construct a multivariate nonparametric smoothing method that avoids bandwidth covariance matrix estimation?
• RQ2 What are the theoretical convergence rates of Fourier kernel-based density and regression estimators compared to standard kernel methods?
• RQ3 Does the product of one-dimensional Fourier kernels in higher dimensions preserve dependence structures inherently, without requiring explicit modeling?
• RQ4 How does the proposed Monte Carlo estimator for multivariate functions perform in terms of bias, variance, and asymptotic normality?
• RQ5 Can the Fourier kernel approach be extended to modal regression and transition density estimation with improved finite-sample performance?

Key findings

• The proposed Fourier kernel estimator achieves faster convergence rates than standard kernel estimators for multivariate density and regression functions.
• The method does not require explicit bandwidth covariance matrix selection, as the product of one-dimensional Fourier kernels inherently captures dependence across dimensions.
• The estimator $ \widehat{m}_{n,R}(x) $ is asymptotically normal with $ \sqrt{n/R^d} \cdot \frac{\widehat{a}_2(x)}{\widehat{f}_{n,R}(x)} \xrightarrow{d} \mathcal{N}\left(0, \frac{\sigma^2}{p_0(x)\pi^d}\right) $, confirming its asymptotic efficiency.
• The bias term $ \sqrt{n/R^d} \cdot \frac{\widehat{a}_1(x)}{\widehat{f}_{n,R}(x)} \xrightarrow{p} 0 $, indicating that bias diminishes under appropriate $ R $-growth conditions.
• Theoretical bounds show that the empirical process deviation is controlled by $ \mathcal{O}_P\left(\sqrt{\frac{R^d \log R}{n}}\right) $, ensuring uniform convergence.
• Numerical illustrations confirm the method's practical viability and superior performance in density and regression estimation compared to classical kernel approaches.