[Paper Review] Efficient online learning with kernels for adversarial large scale problems

Rémi Jézéquel, Pierre Gaillard+1 more|arXiv (Cornell University)|Feb 26, 2019

Advanced Bandit Algorithms Research27 references3 citations

TL;DR

This paper proposes an efficient online kernel learning algorithm that achieves near-optimal regret for large-scale, adversarial datasets by using kernel approximation via basis functions. For the Gaussian kernel, it precomputes basis functions via Taylor expansion, achieving $O((\log n)^{d+1})$ regret with $O((\log n)^{2d})$ per-round complexity, making it suitable when $n \gg e^d$. For general low-effective-dimension kernels, it uses data-adaptive Nyström sampling to maintain computational efficiency while improving on prior online kernel methods.

ABSTRACT

We are interested in a framework of online learning with kernels for low-dimensional but large-scale and potentially adversarial datasets. We study the computational and theoretical performance of online variations of kernel Ridge regression. Despite its simplicity, the algorithm we study is the first to achieve the optimal regret for a wide range of kernels with a per-round complexity of order $n^α$ with $α< 2$. The algorithm we consider is based on approximating the kernel with the linear span of basis functions. Our contributions is two-fold: 1) For the Gaussian kernel, we propose to build the basis beforehand (independently of the data) through Taylor expansion. For $d$-dimensional inputs, we provide a (close to) optimal regret of order $O((\log n)^{d+1})$ with per-round time complexity and space complexity $O((\log n)^{2d})$. This makes the algorithm a suitable choice as soon as $n \gg e^d$ which is likely to happen in a scenario with small dimensional and large-scale dataset; 2) For general kernels with low effective dimension, the basis functions are updated sequentially in a data-adaptive fashion by sampling Nystr{ö}m points. In this case, our algorithm improves the computational trade-off known for online kernel regression.

Motivation & Objective

• Address the challenge of online kernel learning in large-scale, adversarial settings where standard kernel methods suffer from $O(n^2)$ per-round complexity.
• Achieve optimal or near-optimal regret guarantees in online kernel Ridge regression despite adversarial data sequences.
• Reduce computational complexity below $O(n^2)$ per round while maintaining theoretical performance for both Gaussian and general kernels.
• Develop a scalable framework applicable to low-dimensional but high-volume datasets where $n \gg e^d$ is common.
• Provide a unified approach that combines precomputed basis functions for Gaussian kernels and adaptive Nyström sampling for general kernels.

Proposed method

• For Gaussian kernels, construct basis functions in advance using Taylor expansion of the kernel function to enable efficient online updates.
• Use kernel approximation via the linear span of precomputed basis functions to reduce the dimensionality of the online learning problem.
• Apply the Exponentiated Gradient algorithm with a modified kernelized prediction rule to maintain regret bounds while reducing complexity.
• For general kernels with low effective dimension, use sequential Nyström sampling to dynamically select landmark points and maintain a low-rank approximation of the kernel matrix.
• Maintain a recursive update mechanism using Cholesky updates (cholup) to efficiently compute inverse and prediction weights in real time.
• Introduce a modified version of the Kernel-AWV algorithm (PKAWV) with Nyström projections to reduce complexity while preserving regret guarantees.

Experimental results

Research questions

• RQ1 Can we achieve near-optimal regret in online kernel Ridge regression with per-round complexity below $O(n^2)$ for large-scale datasets?
• RQ2 Is it possible to precompute basis functions for the Gaussian kernel such that the resulting algorithm maintains optimal regret and sub-quadratic time complexity?
• RQ3 How can we adaptively select Nyström points in an online fashion to maintain low computational cost and strong regret performance for general kernels?
• RQ4 Can we improve the computational trade-off in online kernel regression without sacrificing theoretical regret guarantees?
• RQ5 Does the use of data-adaptive Nyström sampling lead to better regret bounds than previous approximations, especially in the presence of adversarial data?

Key findings

• For the Gaussian kernel, the proposed method achieves a regret bound of $O((\log n)^{d+1})$, which is close to optimal for this class of kernels.
• The per-round time and space complexity are $O((\log n)^{2d})$, significantly improving over the $O(n^2)$ complexity of standard kernel methods.
• The algorithm remains efficient when $n \gg e^d$, which is typical in low-dimensional, large-scale learning scenarios.
• For general kernels with low effective dimension, the data-adaptive Nyström approach reduces complexity while maintaining regret bounds of the form $\lambda \|f\|^2 + B^2 d_{\text{eff}}(\lambda)$, matching the optimal rate.
• The method avoids the multiplicative factor $m$ and dependency on $C$ seen in prior Nyström-based approximations, leading to more robust regret guarantees.
• The algorithm achieves optimal regret for a wide range of kernels with $\alpha < 2$ in the $n^\alpha$ complexity bound, making it the first such method to do so.