[Paper Review] Quantitative Strong Convergence for the Hybrid Steepest Descent Method

Daniel Körnlein|arXiv (Cornell University)|Oct 3, 2016

Optimization and Variational Analysis13 references3 citations

TL;DR

This paper provides the first quantitative analysis of the Hybrid Steepest Descent Method (HSDM) for solving variational inequality problems in Hilbert spaces, using proof-mining techniques to extract metastability rates and complexity bounds. It establishes strong convergence with explicit rates for both single and finite families of nonexpansive mappings, offering effective bounds on convergence uniformity.

ABSTRACT

We provide new complexity information for the convergence of the Hybrid Steepest Descent Method for solving the Variational Inequality Problem for a strict contraction on Hilbert space over a closed convex set C given either as the fixed point set of a single nonexpansive mapping or the intersection of the fixed point sets of a finite family of nonexpansive mappings. More precisely, we give metastability rates in the sense of Tao for those cases. The results in this paper were extracted from a proof due to Yamada using proof-mining techniques, and provide a thorough quantitative analysis of the Hybrid Steepest Descent Method.

Motivation & Objective

• To provide a quantitative analysis of the Hybrid Steepest Descent Method (HSDM) for solving variational inequality problems (VIP) in Hilbert spaces.
• To extract metastability rates—quantitative convergence information—in the sense of Tao for HSDM under strict contraction and strong monotonicity assumptions.
• To analyze both cases: when the solution set is the fixed point set of a single nonexpansive mapping and when it is the intersection of fixed point sets of a finite family of such mappings.
• To offer complexity information for the HSDM by transforming Yamada's qualitative convergence proof into effective, uniform bounds using proof-theoretic techniques.
• To extend the applicability of HSDM by providing explicit, computable convergence rates without requiring closed-form projections onto the solution set.

Proposed method

• The method applies proof-mining techniques to Yamada's original proof of strong convergence for the HSDM, transforming non-uniform convergence into uniform metastability bounds.
• It derives explicit metastability rates for the HSDM iteration sequence $ u_{n+1} := T(u_n) - \lambda_{n+1}\mu\mathcal{F}(T(u_n)) $, where $ T $ is nonexpansive and $ \mathcal{F} $ is $ \kappa $-Lipschitz and $ \eta $-strongly monotone.
• For the finite family case, the iteration is $ u_{n+1} := T_{[n+1]}(u_n) - \lambda_{n+1}\mu\mathcal{F}(T_{[n+1]}(u_n)) $, with $ [n] = n \mod N $, and convergence is analyzed under summability and vanishing step-size conditions.
• The analysis relies on bounding the distance to the solution set using the fixed point properties of nonexpansive mappings and the strong monotonicity of $ \mathcal{F} $, with explicit control over error terms.
• The resulting bounds are expressed as a function of $ \varepsilon $, $ \kappa $, $ \eta $, and the initial distance to the solution, yielding a computable modulus of metastability.
• The method ensures that for any $ \varepsilon > 0 $, there exists a uniform $ N $ such that for all $ m,n \geq N $, $ \|u_n - u_m\| < \varepsilon $, with $ N $ effectively computable from the problem parameters.

Experimental results

Research questions

• RQ1 What is the quantitative rate of convergence for the Hybrid Steepest Descent Method when solving variational inequalities in Hilbert spaces?
• RQ2 How can metastability rates be extracted from the proof of strong convergence for HSDM using proof-mining techniques?
• RQ3 What are the effective bounds on the number of iterations required to achieve $ \varepsilon $-accuracy in the HSDM under different assumptions on the solution set?
• RQ4 How do the convergence rates differ between the single nonexpansive mapping case and the finite intersection case?
• RQ5 Can the HSDM be analyzed quantitatively without requiring explicit projections onto the solution set?

Key findings

• The paper provides the first explicit metastability rate for the Hybrid Steepest Descent Method, ensuring that for any $ \varepsilon > 0 $, there exists $ N $ such that $ \|u_n - u_m\| < \varepsilon $ for all $ m,n \geq N $, with $ N $ computable from problem parameters.
• For the single nonexpansive mapping case, the convergence rate is quantified under the conditions $ \lambda_n \to 0 $, $ \sum \lambda_n = \infty $, and $ \frac{\lambda_n - \lambda_{n+1}}{\lambda_n^2} \to 0 $, yielding a computable modulus of metastability.
• In the finite family case ($ N \geq 1 $), the method allows for $ \lambda_n = 1/n $, and the convergence rate is again quantified via a modulus of metastability, improving over previous results.
• The analysis shows that the HSDM converges strongly with effective bounds even when the projection onto the solution set is not available, relying only on nonexpansive mappings.
• The convergence rate depends explicitly on the strong monotonicity constant $ \eta $, Lipschitz constant $ \kappa $, and the initial distance to the solution, with explicit dependence on $ \varepsilon $, $ \tau = 1 - \sqrt{1 - \mu(2\eta - \mu\kappa^2)} $, and the diameter of the initial orbit.
• The paper establishes that the HSDM is robust under weak assumptions: even when the solution set is not bounded, convergence rates can be derived under mild initial distance constraints to a fixed point.