[Paper Review] Streaming kernel regression with provably adaptive mean, variance, and regularization

Audrey Durand, Odalric-Ambrym Maillard+1 more|arXiv (Cornell University)|Aug 2, 2017

Advanced Bandit Algorithms Research25 references21 citations

TL;DR

This paper proposes a streaming kernel regression method with provably adaptive regularization, mean, and variance estimation in online settings. By leveraging self-normalized inequalities and adaptive variance bounds, it achieves uniform confidence bounds over time and points, enabling theoretically sound adaptive tuning without assuming known noise variance—demonstrated in kernelized bandits with improved regret bounds.

ABSTRACT

We consider the problem of streaming kernel regression, when the observations arrive sequentially and the goal is to recover the underlying mean function, assumed to belong to an RKHS. The variance of the noise is not assumed to be known. In this context, we tackle the problem of tuning the regularization parameter adaptively at each time step, while maintaining tight confidence bounds estimates on the value of the mean function at each point. To this end, we first generalize existing results for finite-dimensional linear regression with fixed regularization and known variance to the kernel setup with a regularization parameter allowed to be a measurable function of past observations. Then, using appropriate self-normalized inequalities we build upper and lower bound estimates for the variance, leading to Bersntein-like concentration bounds. The later is used in order to define the adaptive regularization. The bounds resulting from our technique are valid uniformly over all observation points and all time steps, and are compared against the literature with numerical experiments. Finally, the potential of these tools is illustrated by an application to kernelized bandits, where we revisit the Kernel UCB and Kernel Thompson Sampling procedures, and show the benefits of the novel adaptive kernel tuning strategy.

Motivation & Objective

• To address the lack of theoretical guarantees in adaptive kernel regression when regularization and noise variance are tuned online.
• To develop confidence bounds that remain valid uniformly over all time steps and input points in streaming kernel regression.
• To enable adaptive regularization based on empirical variance estimates without assuming known noise variance.
• To provide a theoretically grounded method for online tuning of kernel hyperparameters in sequential learning.
• To improve performance in kernelized bandit algorithms by replacing fixed regularization with adaptive, data-driven choices.

Proposed method

• Generalizes finite-dimensional concentration inequalities to kernel regression with regularization as a measurable function of past observations.
• Introduces a streaming variance estimator that provides upper and lower bounds on noise variance using self-normalized inequalities.
• Derives empirical Bernstein-like concentration bounds that incorporate estimated variance, enabling tighter confidence intervals.
• Uses the variance estimates to adaptively tune the regularization parameter at each time step, ensuring theoretical guarantees.
• Applies the framework to kernelized bandits, revising Kernel UCB and Thompson Sampling with adaptive regularization.
• Employs Azuma-Hoeffding inequality to control the sum of estimation errors in the regret analysis, ensuring high-probability bounds.

Experimental results

Research questions

• RQ1 Can we maintain uniform confidence bounds in streaming kernel regression when the regularization parameter is adapted based on past observations?
• RQ2 How can we estimate the noise variance in real-time without prior knowledge, while preserving theoretical guarantees?
• RQ3 What is the impact of adaptive regularization on regret in kernelized bandit algorithms?
• RQ4 Can we derive concentration inequalities for kernel regression with time-varying regularization and estimated variance?
• RQ5 How does the proposed adaptive method compare to fixed-parameter baselines in terms of regret and confidence interval tightness?

Key findings

• The method achieves uniform confidence bounds over all time steps and input points, even with adaptive regularization.
• The adaptive regularization strategy leads to tighter confidence intervals compared to fixed-parameter approaches.
• The regret bound in kernelized bandits scales as O(√T log T) with the adaptive method, improving upon fixed-parameter baselines.
• The framework enables theoretical guarantees in online kernel learning without assuming known noise variance.
• Numerical experiments confirm the method's practical benefits in terms of estimation accuracy and regret reduction.
• The derived bounds are valid under sub-Gaussian noise and do not require prior knowledge of the noise level.