[Paper Review] Local Polynomial Regression Based on Functional Data

Karim Benhenni, David Degras|arXiv (Cornell University)|Jul 20, 2011

Bayesian Methods and Mixture Models26 references3 citations

TL;DR

This paper develops local polynomial regression for functional data, deriving asymptotic bias and variance expressions for estimating regression functions and their derivatives under general error processes. It establishes optimal sampling densities and bandwidths, proves asymptotic normality, and validates performance via simulations comparing exact, asymptotic, and cross-validated bandwidths.

ABSTRACT

Suppose that $n$ statistical units are observed, each following the model $Y(x_j)=m(x_j)+ ε(x_j),\, j=1,...,N,$ where $m$ is a regression function, $0 \leq x_1

Motivation & Objective

• To extend local polynomial regression to functional data settings where entire curves are observed at discrete points.
• To derive asymptotic bias and variance expressions for the estimator of the regression function and its derivatives under general error processes.
• To determine optimal sampling densities and bandwidths that minimize mean squared error in the estimation of m and its derivatives.
• To establish asymptotic normality of the local polynomial estimator for inference purposes.
• To compare the performance of different bandwidth selection methods (exact optimal, asymptotic optimal, cross-validation) via extensive simulations.

Proposed method

• Uses local polynomial fitting of order p to estimate the regression function m(x) and its derivatives at a fixed point x.
• Derives asymptotic bias and variance expressions as n, N → ∞ under differentiability conditions on the error process covariance function.
• Applies Taylor expansions and matrix asymptotics to the design matrix and its inverse, leveraging the sampling density f and kernel K.
• Derives optimal sampling density f*(x) that minimizes the asymptotic variance of the estimator, based on the error process covariance structure.
• Derives optimal bandwidths by minimizing the asymptotic mean squared error, using analytical expressions for bias and variance.
• Proves asymptotic normality of the estimator in the space of continuous functions, enabling simultaneous confidence bands and hypothesis testing.

Experimental results

Research questions

• RQ1 How do the asymptotic bias and variance of local polynomial estimators behave for functional data with dependent or nonstationary errors?
• RQ2 What is the optimal sampling density f*(x) that minimizes the asymptotic variance of the local polynomial estimator?
• RQ3 What are the optimal bandwidths for local polynomial fitting of m and its derivatives under different error covariance structures?
• RQ4 How does the asymptotic normality of the estimator support inference such as confidence bands or hypothesis tests?
• RQ5 How do exact optimal, asymptotic optimal, and cross-validated bandwidths compare in finite-sample performance for derivative estimation?

Key findings

• Asymptotic bias expressions are derived under standard regularity conditions, with variance expressions depending on the error process covariance and its derivatives.
• Optimal sampling density f*(x) is derived as proportional to the square root of the sum of the absolute values of the second-order derivatives of the error covariance function.
• Optimal bandwidths are derived analytically for local constant, linear, quadratic, and cubic fits of m and m′, minimizing asymptotic mean squared error.
• Asymptotic normality is established for the local polynomial estimator in the space of continuous functions, enabling simultaneous confidence bands.
• Simulations show that exact optimal bandwidths yield the best performance in terms of mean squared error, followed by asymptotic optimal and cross-validated bandwidths.
• The variance expansion reveals that the influence of error correlation is captured in the second-order terms involving ρ^{(0,2)}, ρ^{(1,1)}, and ρ^{(0,4)} of the covariance function.