[Paper Review] Nonparametric and Varying Coefficient Modal Regression

Weixin Yao, Sijia Xiang|arXiv (Cornell University)|Feb 22, 2016

Advanced Statistical Methods and Models19 references6 citations

TL;DR

This paper introduces nonparametric modal regression using local polynomial estimation to model the mode of the conditional response distribution, offering robust and efficient prediction under skewed or heavy-tailed data. It establishes asymptotic properties and extends the method to varying coefficient models, demonstrating superior performance over mean and median regression in simulation and real health expenditure data.

ABSTRACT

In this article, we propose a new nonparametric data analysis tool, which we call nonparametric modal regression, to investigate the relationship among interested variables based on estimating the mode of the conditional density of a response variable Y given predictors X. The nonparametric modal regression is distinguished from the conventional nonparametric regression in that, instead of the conditional average or median, it uses the "most likely" conditional values to measures the center. Better prediction performance and robustness are two important characteristics of nonparametric modal regression compared to traditional nonparametric mean regression and nonparametric median regression. We propose to use local polynomial regression to estimate the nonparametric modal regression. The asymptotic properties of the resulting estimator are investigated. To broaden the applicability of the nonparametric modal regression to high dimensional data or functional/longitudinal data, we further develop a nonparametric varying coefficient modal regression. A Monte Carlo simulation study and an analysis of health care expenditure data demonstrate some superior performance of the proposed nonparametric modal regression model to the traditional nonparametric mean regression and nonparametric median regression in terms of the prediction performance.

Motivation & Objective

• To develop a nonparametric method for estimating the conditional mode of a response variable given predictors, avoiding parametric assumptions on the mode function.
• To improve prediction accuracy and robustness in regression by focusing on the 'most likely' value (mode) rather than the mean or median.
• To extend the model to varying coefficient structures for greater flexibility in high-dimensional or functional data settings.
• To establish the asymptotic properties of the proposed estimators, including bias and variance, under regularity conditions.
• To provide a practical estimation algorithm, including an EM-type procedure, for implementation in real data applications.

Proposed method

• Estimates the conditional mode using local polynomial regression, where the mode is identified as the maximizer of a kernel-smoothed conditional density.
• Employs a two-stage kernel density estimator for the joint density $ f(x,y) $, with bandwidths $ h_1 $ and $ h_2 $ for predictors and response, respectively.
• Uses an EM-type algorithm to iteratively update estimates of the mode and associated parameters, improving convergence and stability.
• Derives asymptotic bias and variance expressions for the estimator using Taylor expansions and stochastic approximations under regularity conditions.
• Applies the local polynomial approach to estimate both the mode and its derivatives, enabling inference on the shape of the mode function.
• Extends the model to varying coefficient modal regression by allowing the mode function to depend on covariates through flexible, nonparametric coefficient functions.

Experimental results

Research questions

• RQ1 Can nonparametric modal regression outperform traditional mean and median regression in terms of prediction accuracy under skewed or heavy-tailed error distributions?
• RQ2 What are the asymptotic properties (bias, variance, consistency) of the local polynomial estimator for the conditional mode?
• RQ3 How can the modal regression model be extended to handle varying coefficient structures in high-dimensional or functional data?
• RQ4 What is the finite-sample performance of the proposed estimator in comparison to mean and median regression under model misspecification?
• RQ5 Can the proposed EM-type algorithm reliably estimate the mode function in complex data settings with convergence guarantees?

Key findings

• The proposed nonparametric modal regression estimator achieves lower prediction error than mean and median regression under skewed error distributions, as demonstrated in Monte Carlo simulations.
• Asymptotic bias and variance of the estimator are derived, showing that bias depends on second and third derivatives of the conditional density and bandwidth choices.
• The asymptotic distribution of the estimator is normal, with variance proportional to $ 1/(n h_1 h_2^3) $, indicating convergence rates dependent on bandwidth selection.
• The varying coefficient modal regression model successfully captures complex, non-linear relationships in high-dimensional data, improving model fit and prediction over parametric alternatives.
• Empirical analysis on health care expenditure data shows that modal regression provides more robust and accurate predictions than mean regression, especially in the presence of outliers or skewness.
• The EM-type algorithm improves numerical stability and convergence speed in estimating the mode function, particularly in high-dimensional or complex data settings.