[Paper Review] A simple method to construct confidence bands in functional linear regression

Masaaki Imaizumi, Kengo Kato|arXiv (Cornell University)|Dec 22, 2016

Advanced Statistical Methods and Models42 references3 citations

TL;DR

This paper proposes a simple, asymptotically valid method to construct confidence bands for the slope function in functional linear regression using a principal component analysis (PCA)-based estimator. By relaxing coverage to apply to 'most' points rather than all, it establishes theoretical validity under standard regularity conditions and provides a practical cut-off selection rule for PCA, enabling reliable uncertainty quantification in functional regression.

ABSTRACT

This paper develops a simple method to construct confidence bands, centered at a principal component analysis (PCA) based estimator, for the slope function in a functional linear regression model with a scalar response variable and a functional predictor variable. The PCA-based estimator is a series estimator with estimated basis functions, and so construction of valid confidence bands for it is a non-trivial challenge. We propose a confidence band that aims at covering the slope function at "most" of points with a prespecified probability (level), and prove its asymptotic validity under suitable regularity conditions. Importantly, this is the first paper that derives confidence bands having theoretical justifications for the PCA-based estimator. We also propose a practical method to choose the cut-off level used in PCA-based estimation, and conduct numerical studies to verify the finite sample performance of the proposed confidence band. Finally, we apply our methodology to spectrometric data, and discuss extensions of our methodology to cases where additional vector-valued regressors are present.

Motivation & Objective

• To address the lack of theoretically justified confidence bands for PCA-based estimators in functional linear regression, a critical gap in the literature.
• To develop a confidence band that covers the true slope function at 'most' points with a prespecified probability, rather than uniformly across all points.
• To provide a practical method for selecting the PCA cut-off level that ensures undersmoothing and asymptotic validity.
• To verify finite-sample performance through numerical studies and apply the method to real spectrometric data.
• To extend the methodology to settings with additional vector-valued covariates.

Proposed method

• Proposes a confidence band centered at the PCA-based estimator of the slope function, using a relaxation of coverage to apply to a majority of points in the domain.
• Employs a stochastic approximation approach to account for the randomness in estimated eigenfunctions from the empirical covariance operator.
• Uses a multiplier central limit theorem for high-dimensional, heteroscedastic, and dependent random vectors to justify the asymptotic distribution of the estimator's deviation.
• Derives a bound on the sup-norm deviation of the estimator by controlling the third absolute moment of the weighted sum of empirical coefficients.
• Introduces a cut-off level selection rule that slightly exceeds the optimal level minimizing estimated $L^2$-risk, ensuring undersmoothing.
• Applies a concentration inequality and moment bounds under moment and eigenvalue decay conditions to control the estimation error of the eigenfunctions.

Experimental results

Research questions

• RQ1 Can a confidence band be constructed for the PCA-based estimator in functional linear regression that is theoretically justified and practically implementable?
• RQ2 How can the randomness in the estimated eigenfunctions from the empirical covariance operator be properly accounted for in inference?
• RQ3 What is a practical and theoretically sound method for selecting the PCA cut-off level to ensure valid coverage?
• RQ4 Does the proposed confidence band maintain asymptotic validity under standard regularity conditions in functional linear regression?
• RQ5 How does the method perform in finite samples, and can it be extended to models with additional vector-valued regressors?

Key findings

• The proposed confidence band is asymptotically valid under standard regularity conditions, covering the true slope function at 'most' points with a prespecified probability level.
• The method achieves $o_P(1)$ convergence in the sup-norm deviation bound under Condition (14), ensuring the band's validity as sample size increases.
• The cut-off level selection rule, based on slightly oversmoothing the optimal $L^2$-risk minimizing level, ensures undersmoothing and theoretical validity.
• Numerical studies confirm the finite-sample performance of the confidence band, demonstrating appropriate coverage rates.
• The method is extended to models with additional vector-valued regressors, broadening its applicability.
• Theoretical justification relies on bounding the third absolute moment of a weighted sum of empirical coefficients, leading to a $O_P(m_n^{7/4}/n^{1/2 - 1/(2q)})$ error term that vanishes under regularity conditions.