[Paper Review] Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain

Dmitrii M. Ostrovskii, Babak Barazandeh+1 more|arXiv (Cornell University)|Oct 8, 2021

Sparse and Compressive Sensing Techniques101 references4 citations

TL;DR

This paper proposes a novel approach to solving nonconvex-nonconcave min-max optimization problems by approximating the inner maximization via high-order Taylor expansions in $ y $, under the key assumption that the maximization domain $ Y $ has small diameter. The main contribution is a theoretical guarantee that, when the diameter of $ Y $ is $ O(\varepsilon^{2/(k+1)}) $ for $ k $-th order approximation, any $ \varepsilon $-stationary point in the surrogate problem remains $ O(\varepsilon) $-stationary in the original problem, with nearly optimal bounds shown to be necessary.

ABSTRACT

We study the problem of finding approximate first-order stationary points in optimization problems of the form $\min_{x \in X} \max_{y \in Y} f(x,y)$, where the sets $X,Y$ are convex and $Y$ is compact. The objective function $f$ is smooth, but assumed neither convex in $x$ nor concave in $y$. Our approach relies upon replacing the function $f(x,\cdot)$ with its $k$th order Taylor approximation (in $y$) and finding a near-stationary point in the resulting surrogate problem. To guarantee its success, we establish the following result: let the Euclidean diameter of $Y$ be small in terms of the target accuracy $\varepsilon$, namely $O(\varepsilon^{\frac{2}{k+1}})$ for $k \in \mathbb{N}$ and $O(\varepsilon)$ for $k = 0$, with the constant factors controlled by certain regularity parameters of $f$; then any $\varepsilon$-stationary point in the surrogate problem remains $O(\varepsilon)$-stationary for the initial problem. Moreover, we show that these upper bounds are nearly optimal: the aforementioned reduction provably fails when the diameter of $Y$ is larger. For $0 \le k \le 2$ the surrogate function can be efficiently maximized in $y$; our general approximation result then leads to efficient algorithms for finding a near-stationary point in nonconvex-nonconcave min-max problems, for which we also provide convergence guarantees.

Motivation & Objective

• To address the challenge of finding approximate first-order stationary points in nonconvex-nonconcave min-max problems where $ f $ is neither convex in $ x $ nor concave in $ y $.
• To develop a tractable algorithmic framework for such problems by leveraging high-order Taylor approximations of $ f(x, \cdot) $ in the $ y $-variable.
• To establish theoretical conditions under which the surrogate problem yields near-stationary solutions for the original problem, particularly focusing on the role of the diameter of $ Y $.
• To show that the required small diameter condition is nearly optimal, as the reduction fails when the diameter exceeds the derived threshold.
• To provide efficient algorithms for $ k = 0,1,2 $ with convergence guarantees, enabling practical application in machine learning contexts like GANs and robust training.

Proposed method

• Replace $ f(x, \cdot) $ with its $ k $-th order Taylor approximation in $ y $, forming a surrogate min-max problem that is easier to solve.
• Use a small diameter of $ Y $, specifically $ O(\varepsilon^{2/(k+1)}) $ for $ k \in \mathbb{N} $ and $ O(\varepsilon) $ for $ k=0 $, to ensure the surrogate solution approximates the original problem's stationary points.
• Establish a theoretical bound showing that any $ \varepsilon $-stationary point in the surrogate problem is $ O(\varepsilon) $-stationary in the original problem, with constants depending on regularity parameters of $ f $.
• For $ k \leq 2 $, the surrogate problem can be efficiently maximized in $ y $, enabling the design of practical algorithms with convergence guarantees.
• Leverage the Lanczos method to efficiently solve the inner maximization subproblem in the surrogate, using $ O(m) $ matrix-vector products and $ O(md_{\textsf{y}}) $ memory, where $ m $ is the number of Lanczos iterations.
• Use QR-based orthogonalization and block tridiagonal structure to maintain numerical stability and enable efficient computation of the Hessian approximation in the Krylov subspace.

Experimental results

Research questions

• RQ1 Under what conditions on the diameter of $ Y $ does the $ k $-th order Taylor approximation of $ f(x, \cdot) $ preserve $ \varepsilon $-stationarity in the original min-max problem?
• RQ2 Is the required diameter scaling of $ O(\varepsilon^{2/(k+1)}) $ nearly optimal, or can it be improved?
• RQ3 Can the resulting surrogate problem be efficiently solved for $ k \leq 2 $, and what convergence guarantees can be established?
• RQ4 How do regularity parameters of $ f $ influence the accuracy of the approximation and the required diameter of $ Y $?
• RQ5 Can the proposed method be implemented efficiently in practice, particularly in high-dimensional settings common in machine learning?

Key findings

• The diameter of $ Y $ must be $ O(\varepsilon^{2/(k+1)}) $ for $ k \in \mathbb{N} $, and $ O(\varepsilon) $ for $ k=0 $, to ensure that any $ \varepsilon $-stationary point in the surrogate problem is $ O(\varepsilon) $-stationary in the original problem.
• The required diameter condition is nearly optimal: the reduction provably fails when the diameter exceeds the derived threshold, establishing tightness of the bound.
• For $ k = 0,1,2 $, the surrogate problem can be efficiently maximized in $ y $, enabling the design of practical algorithms with convergence guarantees.
• The computational cost of the algorithm is $ O(m(d_{\textsf{y}} + \text{time}(\text{HVP}))) $, where $ m $ is the number of Lanczos iterations and $ \text{time}(\text{HVP}) $ is the cost of computing Hessian-vector products.
• Memory usage is $ O(md_{\textsf{y}} + \text{mem}(\text{HVP})) $, which can be reduced to $ O(m + d_{\textsf{y}}) $ if only the optimal value is needed, not the solution vector.
• The method ensures that the solution to the surrogate problem remains a valid $ O(\varepsilon) $-stationary point in the original problem, provided the domain $ Y $ is sufficiently small relative to the target accuracy $ \varepsilon $.