[Paper Review] Learning-to-Learn Stochastic Gradient Descent with Biased Regularization

Giulia Denevi, Carlo Ciliberto+2 more|arXiv (Cornell University)|Mar 25, 2019

Stochastic Gradient Optimization Techniques43 citations

TL;DR

The paper introduces a learning-to-learn framework where SGD on biased regularized risk is learned online to exploit task relatedness; it provides excess transfer risk bounds and an online meta-algorithm to estimate the bias.

ABSTRACT

We study the problem of learning-to-learn: inferring a learning algorithm that works well on tasks sampled from an unknown distribution. As class of algorithms we consider Stochastic Gradient Descent on the true risk regularized by the square euclidean distance to a bias vector. We present an average excess risk bound for such a learning algorithm. This result quantifies the potential benefit of using a bias vector with respect to the unbiased case. We then address the problem of estimating the bias from a sequence of tasks. We propose a meta-algorithm which incrementally updates the bias, as new tasks are observed. The low space and time complexity of this approach makes it appealing in practice. We provide guarantees on the learning ability of the meta-algorithm. A key feature of our results is that, when the number of tasks grows and their variance is relatively small, our learning-to-learn approach has a significant advantage over learning each task in isolation by Stochastic Gradient Descent without a bias term. We report on numerical experiments which demonstrate the effectiveness of our approach.

Motivation & Objective

• Motivate learning-to-learn (LTL) as a meta-learning approach to select inner learning algorithms for task families drawn from an environment.
• Propose SGD on a biased regularized true risk as the inner algorithm to leverage task relatedness.
• Derive excess transfer risk bounds showing when a bias improves performance over independent task learning (ITL).
• Develop an online meta-algorithm that incrementally estimates the bias from a sequence of tasks with low space and time complexity.
• Provide theoretical guarantees for the meta-algorithm and demonstrate empirical effectiveness on synthetic and real data.

Proposed method

• Define the inner algorithm as SGD applied to the biased regularized true risk, with bias h and regularization parameter lambda.
• Prove an excess transfer risk bound for a fixed bias h, showing improvements when Var_h^2 is small.
• Introduce a proxy objective L_Zn(h) = min_w R_{Z_n,h}(w) and show it is convex and lambda-smooth with a gradient given by nabla L_Zn(h) = -lambda (w_h(Z_n) - h).
• Develop Algorithm 2 that performs SGD on the meta-objective using epsilon-subgradients derived from the last inner-iteration of SGD, enabling online bias updates.
• Provide a bound for the bias estimate bar{h}_T and the resulting excess transfer risk, including an O(Var_m / sqrt(n)) term and an O(1/sqrt(T)) term.

Experimental results

Research questions

• RQ1 Under what conditions does SGD with a biased regularizer outperform SGD without a bias on related tasks?
• RQ2 How can one online-estimate an optimal bias to minimize transfer risk across a sequence of tasks?
• RQ3 What are the statistical guarantees for an online meta-algorithm updating the bias in terms of excess transfer risk?
• RQ4 How does the proposed approach scale in space/time while preserving theoretical guarantees?
• RQ5 Do empirical results on synthetic and real data support the theoretical benefits of biased regularization in LTL?

Key findings

• When task weight vectors have small variance across tasks, SGD with the right bias yields lower transfer risk than unbiased learning.
• An online meta-algorithm can estimate the bias with low space and time complexity while maintaining statistical guarantees.
• The excess transfer risk bound for a fixed bias shows dependence on Var_h, R, L, and n, with improved performance when Var_h is small.
• The meta-algorithm achieves a transfer-risk bound combining a term decaying with n and another decaying with T, indicating benefits as more tasks are observed.
• Corollaries show ITL (h=0) and an oracle bias scenario provide concrete bounds consistent with LTL literature.
• Experiments on synthetic and real data demonstrate practical effectiveness of the online LTL approach with biased regularization.