[Paper Review] Boosting Functional Regression Models with FDboost

Sarah Brockhaus, David Rügamer+1 more|arXiv (Cornell University)|May 30, 2017

Statistical Methods and Inference59 references11 citations

TL;DR

This paper introduces FDboost, an R package that enables flexible, high-dimensional functional regression modeling via model-based boosting, supporting scalar-on-function, function-on-scalar, and function-on-function regression with mean, quantile, and GAMLSS models. It demonstrates its capabilities through real-world applications in fossil fuel spectrometry and bioelectrical signal analysis, offering advanced features like factor-specific historical effects and robust tuning via resampling.

ABSTRACT

The R add-on package FDboost is a flexible toolbox for the estimation of functional regression models by model-based boosting. It provides the possibility to fit regression models for scalar and functional response with effects of scalar as well as functional covariates, i.e., scalar-on-function, function-on-scalar and function-on-function regression models. In addition to mean regression, quantile regression models as well as generalized additive models for location scale and shape can be fitted with FDboost. Furthermore, boosting can be used in high-dimensional data settings with more covariates than observations. We provide a hands-on tutorial on model fitting and tuning, including the visualization of results. The methods for scalar-on-function regression are illustrated with spectrometric data of fossil fuels and those for functional response regression with a data set including bioelectrical signals for emotional episodes.

Motivation & Objective

• To develop a comprehensive, user-friendly R package for fitting diverse functional regression models using model-based boosting.
• To address the challenge of high-dimensional functional data with variable selection and shrinkage in functional regression.
• To extend boosting to handle complex functional response models, including historical effects and interaction structures.
• To provide practical tools for model tuning, visualization, and uncertainty quantification via bootstrap and stability selection.

Proposed method

• Model-based boosting is used to iteratively combine simple base-learners, minimizing a loss function to estimate functional regression models.
• Base-learners for scalar and functional covariates are implemented with appropriate identifiability constraints and smoothing penalties.
• The package supports mean regression, quantile regression, and GAMLSS by minimizing different loss functions, enabling flexible modeling of response distribution parameters.
• Resampling techniques such as k-fold cross-validation and leave-one-subject-out are used for model tuning and optimal stopping iteration estimation.
• The hmatrix object enables modeling of historical effects by linking past functional covariates to current responses through time-limited integrals.
• Factor-specific historical effects are modeled using the row-wise tensor product operator %X% and anisotropic penalties via the %A%-operator for time-dependent effects.

Experimental results

Research questions

• RQ1 How can model-based boosting be effectively adapted to fit functional regression models with scalar and functional responses in high-dimensional settings?
• RQ2 What is the performance of FDboost in predicting scalar outcomes from functional predictors, such as heat value from spectral data?
• RQ3 Can FDboost model complex functional response data with time-dependent, interaction, and subject-specific effects, such as EMG signals influenced by experimental conditions?
• RQ4 How can optimal model tuning and stopping iterations be achieved in functional regression using resampling techniques?
• RQ5 To what extent can FDboost support advanced modeling features like factor-specific historical effects and anisotropic smoothing in functional data analysis?

Key findings

• FDboost successfully models scalar-on-function regression using spectrometric data of fossil fuels, achieving optimal stopping at 200 boosting iterations after resampling.
• The package enables fitting function-on-scalar and function-on-function regression models on bioelectrical signal data, including complex interaction and historical effects.
• A complex model with three-way interactions and subject-specific effects was fitted in approximately 75 minutes using 15GB of RAM, demonstrating scalability.
• The use of hmatrix and the %X% operator allows for the first implementation of factor-specific historical effects in functional regression, a novel capability.
• Resampling with leave-one-subject-out cross-validation provided stable tuning results, with optimal stopping estimated at 200 iterations after grid search.
• The package supports flexible modeling of all distribution parameters via GAMLSS and enables uncertainty quantification through bootstrap and stability selection.