[Paper Review] Will Bilevel Optimizers Benefit from Loops

Kaiyi Ji, Mingrui Liu+2 more|arXiv (Cornell University)|May 27, 2022

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper provides a unified convergence analysis for bilevel optimizers AID-BiO and ITD-BiO across all loop implementation variants (with or without loops for inner solution and Hessian-inverse approximation). It shows that loops in AID-BiO improve overall efficiency by reducing gradient complexity, while in ITD-BiO, loops are essential to achieve vanishing convergence error—no-loop schemes suffer from unavoidable non-vanishing error, validated by theory and experiments.

ABSTRACT

Bilevel optimization has arisen as a powerful tool for solving a variety of machine learning problems. Two current popular bilevel optimizers AID-BiO and ITD-BiO naturally involve solving one or two sub-problems, and consequently, whether we solve these problems with loops (that take many iterations) or without loops (that take only a few iterations) can significantly affect the overall computational efficiency. Existing studies in the literature cover only some of those implementation choices, and the complexity bounds available are not refined enough to enable rigorous comparison among different implementations. In this paper, we first establish unified convergence analysis for both AID-BiO and ITD-BiO that are applicable to all implementation choices of loops. We then specialize our results to characterize the computational complexity for all implementations, which enable an explicit comparison among them. Our result indicates that for AID-BiO, the loop for estimating the optimal point of the inner function is beneficial for overall efficiency, although it causes higher complexity for each update step, and the loop for approximating the outer-level Hessian-inverse-vector product reduces the gradient complexity. For ITD-BiO, the two loops always coexist, and our convergence upper and lower bounds show that such loops are necessary to guarantee a vanishing convergence error, whereas the no-loop scheme suffers from an unavoidable non-vanishing convergence error. Our numerical experiments further corroborate our theoretical results.

Motivation & Objective

• To provide a unified convergence analysis for AID-BiO and ITD-BiO across all loop implementation choices.
• To characterize the computational complexity of all implementations and enable rigorous comparison.
• To determine whether loops in inner-level solution estimation and Hessian-inverse approximation improve overall efficiency.
• To establish theoretical bounds showing loops are necessary for vanishing error in ITD-BiO.
• To validate theoretical findings with numerical experiments.

Proposed method

• Derives unified convergence bounds for AID-BiO and ITD-BiO under varying loop configurations (N-loop, Q-loop, no-loop, N-Q-loop).
• Uses gradient descent to solve linear systems in all implementations for fair comparison.
• Applies theoretical analysis to derive upper and lower bounds on gradient and matrix-vector product complexities.
• Constructs a specific loss function example to prove lower bounds for ITD-BiO, showing non-vanishing error in no-loop case.
• Employs telescoping techniques and matrix norm analysis to derive convergence rates.
• Validates theoretical results with numerical experiments on bilevel optimization tasks.

Experimental results

Research questions

• RQ1 Does using loops in the inner-level solution estimation step improve computational efficiency in AID-BiO?
• RQ2 Is the loop for approximating the Hessian-inverse-vector product beneficial in AID-BiO?
• RQ3 Are loops necessary in ITD-BiO to achieve vanishing convergence error?
• RQ4 What is the computational complexity of each implementation variant of AID-BiO and ITD-BiO?
• RQ5 Can a no-loop scheme for ITD-BiO achieve convergence to a stationary point?

Key findings

• For AID-BiO, using loops in the inner-level solution estimation (N-loop) reduces overall gradient complexity, despite higher per-step cost.
• In AID-BiO, the Q-loop for Hessian-inverse approximation reduces gradient complexity, improving efficiency.
• For ITD-BiO, the no-loop implementation suffers from an unavoidable non-vanishing convergence error, making loops necessary.
• Theoretical upper and lower bounds confirm that loops in ITD-BiO are essential to achieve vanishing error.
• Numerical experiments corroborate the theoretical findings, showing superior convergence with loops in both AID-BiO and ITD-BiO.
• The computational complexity of the N-Q-loop AID-BiO variant is O~(κ⁴ε⁻¹) for both gradient and matrix-vector products, outperforming no-loop variants.