[Paper Review] A Hölderian backtracking method for min-max and min-min problems

Jérôme Bolte, Lilian E. Glaudin+2 more|arXiv (Cornell University)|Jul 17, 2020

Stochastic Gradient Optimization Techniques35 references4 citations

TL;DR

This paper introduces a novel backtracking gradient method for min-max and min-min optimization problems, leveraging Hölderian regularity of the value function to enable automatic, adaptive step sizes. Under semi-algebraic and uniqueness assumptions, the method achieves convergence to critical points with a global rate of $ O(\epsilon^{-[2+c]}) $, where $ c $ is negligible, and demonstrates strong performance on GANs and optimal transport problems.

ABSTRACT

We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple GAN problems obtaining promising numerical results.

Motivation & Objective

• To develop a robust, adaptive optimization algorithm for non-convex min-max and min-min problems arising in adversarial learning and game theory.
• To establish theoretical convergence guarantees under minimal assumptions, particularly semi-algebraic structure and unique best response.
• To enable aggressive step sizes through hidden Hölder regularity of the value function, avoiding conservative backtracking.
• To provide a practical algorithm with automatic line search that maintains sufficient decrease without monotonicity.
• To validate the method numerically on challenging problems such as Sinkhorn GANs and Wasserstein GANs.

Proposed method

• The method treats the min-max problem as a gradient descent on the value function $ g(x) = \max_y L(x,y) $, exploiting its locally Hölderian gradient under semi-algebraic assumptions.
• It introduces a non-monotone backtracking strategy that adaptively adjusts the step size $ \gamma_n $ based on a sufficient decrease condition involving $ \|\nabla_x L(x_n, y_n)\|^{\rho k} $.
• The step size is updated via $ \gamma_n(x_n) = \alpha^k \min\{1, \|\nabla_x L(x_n, y_n)\|^{\rho k}\} \gamma $, where $ \alpha \in (0,1) $, allowing large steps when gradient decay is slow.
• A key innovation is the use of the hidden regularity of $ g $ to justify large steps, departing from standard cautious backtracking.
• The algorithm maintains sufficient decrease without requiring monotonicity of the backtracking index $ k_n $, improving efficiency.
• For problems where exact argmax is infeasible, such as in Wasserstein GANs, the method uses gradient ascent approximations $ y_n \approx \arg\max_y L(x_n, y) $.

Experimental results

Research questions

• RQ1 Can we design a gradient-based method for min-max problems that automatically adapts step sizes using intrinsic regularity of the value function?
• RQ2 Under what conditions does the value function $ g(x) = \max_y L(x,y) $ have a locally Hölderian gradient?
• RQ3 Can we achieve global convergence and non-asymptotic rates for min-max problems without strong convexity or smoothness assumptions?
• RQ4 How does automatic step size adaptation compare to classical backtracking or constant step size methods in practice?
• RQ5 What is the impact of diagonal backtracking on convergence rate and practical efficiency?

Key findings

• The Backtrack Hölder method achieves a global convergence rate of $ O(\epsilon^{-[2+c]}) $, where $ c $ is a small cost from the diagonal backtracking process.
• Convergence is guaranteed to critical points $ (x^*, y^*) $ satisfying $ y^* = \arg\max_y L(x^*, y) $ and $ \nabla_x L(x^*, y^*) = 0 $, under semi-algebraic and uniqueness assumptions.
• The method provides theoretical justification for using large step sizes by exploiting the locally Hölderian property of the value function's gradient.
• Numerical experiments on Sinkhorn GANs and Wasserstein GANs show promising performance, outperforming constant step size and standard backtracking.
• The algorithm maintains sufficient decrease without monotonicity of the backtracking index, improving practical efficiency.
• A byproduct of the analysis is a global convergence result for Hölder gradient methods, extending prior work in non-convex optimization.