[Paper Review] On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operators

Cong D. Dang, Guanghui Lan|arXiv (Cornell University)|Nov 12, 2013

Optimization and Variational Analysis22 references4 citations

TL;DR

This paper introduces non-Euclidean extragradient (N-EG) methods for solving generalized monotone variational inequalities (GMVI) with non-monotone or pseudo-monotone operators, using a novel residual-based termination criterion and prox-mapping. It establishes iteration complexity bounds of O(1/√k) for Lipschitz continuous operators and demonstrates superior performance over existing methods in large-scale numerical tests.

ABSTRACT

In this paper, we study a class of generalized monotone variational inequality (GMVI) problems whose operators are not necessarily monotone (e.g., pseudo-monotone). We present non-Euclidean extragradient (N-EG) methods for computing approximate strong solutions of these problems, and demonstrate how their iteration complexities depend on the global Lipschitz or Hölder continuity properties for their operators and the smoothness properties for the distance generating function used in the N-EG algorithms. We also introduce a variant of this algorithm by incorporating a simple line-search procedure to deal with problems with more general continuous operators. Numerical studies are conducted to illustrate the significant advantages of the developed algorithms over the existing ones for solving large-scale GMVI problems.

Motivation & Objective

• Address the lack of complexity analysis for extragradient methods in generalized monotone variational inequalities (GMVI), where operators are not necessarily monotone.
• Develop a new termination criterion based on the residual function associated with prox-mapping to ensure convergence to approximate strong solutions.
• Analyze the iteration complexity of N-EG methods under global Lipschitz or Hölder continuity of the operator and smoothness of the distance-generating function.
• Introduce a line-search variant of the N-EG method to handle more general continuous operators beyond Hölder continuity.
• Demonstrate the practical advantages of the proposed algorithms through extensive numerical experiments on large-scale GMVI problems.

Proposed method

• Propose a non-Euclidean extragradient (N-EG) method using prox-mapping instead of standard Euclidean projections, enabling better conditioning for structured problems.
• Define a residual-based termination criterion tied to the prox-mapping, which ensures that small residual implies small optimality gap for approximate strong solutions.
• Establish convergence via a novel complexity analysis showing O(1/√k) iteration complexity for Lipschitz continuous operators, with dependence on operator Lipschitz constant L, smoothness parameter Ω, and algorithm parameters.
• Introduce a line-search procedure to adaptively adjust step-sizes, extending applicability to operators that are continuous but not necessarily Hölder continuous.
• Utilize distance-generating functions with varying smoothness (e.g., p-norm, entropy) to define non-Euclidean metrics, enabling tailored optimization for different problem structures.
• Implement and compare the N-EG method across Euclidean, p-norm, and entropy-based algorithms on benchmark and randomly generated GMVI instances.

Experimental results

Research questions

• RQ1 Can extragradient methods achieve iteration complexity bounds for generalized monotone variational inequalities (GMVI) when the operator is not monotone?
• RQ2 How does the choice of distance-generating function (e.g., p-norm, entropy) affect the convergence rate and practical performance of N-EG methods?
• RQ3 What is the relationship between the residual function and the optimality gap in the context of approximate strong solutions for GMVI?
• RQ4 Can a line-search variant of the N-EG method extend convergence guarantees to operators that are continuous but not Hölder continuous?
• RQ5 How do the proposed N-EG methods compare in practice to standard Euclidean extragradient methods for large-scale GMVI problems?

Key findings

• The N-EG method achieves an iteration complexity of O(1/√k) for solving GMVI problems when the operator is globally Lipschitz continuous, with the bound depending on the operator’s Lipschitz constant L, the smoothness parameter Ω, and algorithm parameters.
• The proposed residual-based termination criterion ensures that a small residual implies a small optimality gap g(y) = max_{z∈X} ⟨F(y), y−z⟩, thus guaranteeing approximate strong solutions.
• Numerical results show that the non-Euclidean N-EG method significantly outperforms the standard Euclidean extragradient method in terms of CPU time and number of projection calls, especially for large-scale problems.
• For the Sun problem and HP Hard problem, the p-norm and entropy-based N-EG variants required substantially fewer projection calls (np) and less CPU time than the Euclidean version, with improvements up to 5–10× in efficiency.
• The line-search variant of the N-EG method successfully handles non-Hölder continuous operators and maintains convergence, demonstrating robustness beyond standard continuity assumptions.
• In large-scale instances (n=8,000), the entropy-based N-EG method achieved convergence in 20.5k CPU seconds with 22,200 projection calls, while the Euclidean method required over 1,700 seconds and 10,761 calls, indicating a clear performance advantage.