[Paper Review] Randomized Stochastic Variance-Reduced Methods for Multi-Task Stochastic Bilevel Optimization

Zhishuai Guo, Quanqi Hu+2 more|arXiv (Cornell University)|May 5, 2021

Stochastic Gradient Optimization Techniques35 references4 citations

TL;DR

This paper proposes randomized stochastic variance-reduced algorithms for non-convex multi-task stochastic bilevel optimization, achieving a sample complexity of $O(1/\\/epsilon^3)$ for single-task problems and $O(m/\\/epsilon^3)$ for multi-task problems by processing only a constant number of lower-level problems per iteration. The method matches the state-of-the-art sample complexity for non-convex stochastic optimization and is the first to address multi-task bilevel problems with optimal convergence rates.

ABSTRACT

In this paper, we consider non-convex stochastic bilevel optimization (SBO) problems that have many applications in machine learning. Although numerous studies have proposed stochastic algorithms for solving these problems, they are limited in two perspectives: (i) their sample complexities are high, which do not match the state-of-the-art result for non-convex stochastic optimization; (ii) their algorithms are tailored to problems with only one lower-level problem. When there are many lower-level problems, it could be prohibitive to process all these lower-level problems at each iteration. To address these limitations, this paper proposes fast randomized stochastic algorithms for non-convex SBO problems. First, we present a stochastic method for non-convex SBO with only one lower problem and establish its sample complexity of $O(1/ε^3)$ for finding an $ε$-stationary point under Lipschitz continuous conditions of stochastic oracles, matching the lower bound for stochastic smooth non-convex optimization. Second, we present a randomized stochastic method for non-convex SBO with $m>1$ lower level problems (multi-task SBO) by processing a constant number of lower problems at each iteration, and establish its sample complexity no worse than $O(m/ε^3)$, which could be a better complexity than that of simply processing all $m$ lower problems at each iteration. Lastly, we establish even faster convergence results for gradient-dominant functions. To the best of our knowledge, this is the first work considering multi-task SBO and developing state-of-the-art sample complexity results.

Motivation & Objective

• To address high sample complexity in existing stochastic bilevel optimization algorithms, which do not match the $O(1/\epsilon^3)$ lower bound for non-convex stochastic optimization.
• To overcome the limitation of existing algorithms that process all $m$ lower-level problems at each iteration, which becomes prohibitive when $m$ is large.
• To develop a randomized stochastic method that processes only a constant number of lower-level problems per iteration while maintaining optimal convergence rates.
• To establish theoretical sample complexity bounds for both single-task and multi-task non-convex stochastic bilevel problems under Lipschitz continuous stochastic oracles.
• To extend the framework to gradient-dominant functions and achieve even faster convergence rates.

Proposed method

• Introduces a stochastic variance-reduced method (SVRB) for single-task non-convex SBO that uses recursive gradient estimators to reduce variance in Hessian and Jacobian approximations.
• Proposes a randomized stochastic algorithm (RSVRB) for multi-task SBO that samples a constant number of lower-level problems per iteration, reducing computational cost while maintaining convergence.
• Employs a recursive variance-reduction technique for estimating the Hessian and Jacobian matrices of the lower-level problem, improving stability and convergence.
• Uses a single-loop, single-timescale update scheme with adaptive step sizes to avoid the need for large mini-batch sizes or double-loop structures.
• Applies conjugate gradient methods to efficiently compute Hessian-vector products in the update of the lower-level variables.
• Introduces a randomized sampling strategy over the $m$ lower-level problems to balance exploration and computational efficiency.

Experimental results

Research questions

• RQ1 Can a stochastic bilevel optimization algorithm achieve the optimal $O(1/\epsilon^3)$ sample complexity for non-convex problems, matching the lower bound for standard non-convex stochastic optimization?
• RQ2 Can a bilevel algorithm be designed to process only a constant number of lower-level problems per iteration, even when $m \gg 1$, without degrading convergence rate?
• RQ3 How does the sample complexity of the proposed method scale with the number of lower-level problems $m$ in the multi-task setting?
• RQ4 Can the proposed method achieve faster convergence for gradient-dominant functions compared to general non-convex cases?
• RQ5 Is the proposed algorithm scalable and efficient in practice, especially when handling large numbers of lower-level tasks?

Key findings

• The proposed SVRB method achieves a sample complexity of $O(1/\epsilon^3)$ for single-task non-convex SBO, matching the known lower bound for non-convex stochastic optimization.
• For multi-task SBO with $m$ lower-level problems, the RSVRB method achieves a sample complexity of $O(m/\epsilon^3)$, which is better than processing all $m$ problems at each iteration when $m$ is large.
• The algorithm outperforms existing baselines such as STABLE, TTSA, and BSA in both sample complexity and running time on logistic regression and hyperparameter optimization tasks.
• In experiments on UCI Adult and web page classification datasets, RSVRB converges faster than STABLE and other baselines in terms of both objective value and running time.
• On multi-task hyperparameter optimization with 200 or 500 lower-level problems, RE-RSVRB significantly outperforms baselines in convergence speed and stability.
• The method maintains strong performance even when the number of lower-level problems increases, demonstrating scalability and robustness in large-scale settings.