[Paper Review] Local convergence for alternating and averaged nonconvex projections

Adrian S. Lewis, D. Lüke+1 more|ArXiv.org|Sep 2, 2007

Sparse and Compressive Sensing Techniques31 references5 citations

TL;DR

This paper establishes local linear convergence for alternating and averaged nonconvex projection methods under the condition of strongly regular intersection, showing that convergence occurs at a rate governed by a regularity modulus when one set is prox-regular or satisfies similar geometric conditions. The results extend classical convergence theory to nonconvex settings with practical implications for feasibility problems in signal processing and optimization.

ABSTRACT

The idea of a finite collection of closed sets having "strongly regular intersection" at a given point is crucial in variational analysis. We show that this central theoretical tool also has striking algorithmic consequences. Specifically, we consider the case of two sets, one of which we assume to be suitably "regular" (special cases being convex sets, smooth manifolds, or feasible regions satisfying the Mangasarian-Fromovitz constraint qualification). We then prove that von Neumann's method of "alternating projections" converges locally to a point in the intersection, at a linear rate associated with a modulus of regularity. As a consequence, in the case of several arbitrary closed sets having strongly regular intersection at some point, the method of "averaged projections" converges locally at a linear rate to a point in the intersection. Inexact versions of both algorithms also converge linearly.

Motivation & Objective

• To establish local linear convergence for nonconvex projection algorithms when the intersection of sets is strongly regular.
• To extend classical convergence results from convex to nonconvex settings using metric regularity and prox-regularity.
• To demonstrate that only one set needs strong geometric regularity (e.g., convex, smooth, or prox-regular) for convergence.
• To show inexact versions of the algorithms also converge linearly, enhancing practical applicability.
• To provide an algorithmic proof of the exact extremal principle via averaged projections.

Proposed method

• Uses von Neumann’s alternating projection method as a foundation, reformulating it in a product space to analyze averaged projections.
• Applies the concept of metric regularity and the regularity modulus to bound convergence rates.
• Employs the normal cone definition and variational analysis tools without relying on advanced machinery.
• Reduces the averaged projection method to an equivalent alternating projection scheme on two sets in a higher-dimensional space.
• Uses prox-regularity and amenability as sufficient conditions for the required geometric regularity.
• Employs numerical experiments with random matrices to illustrate convergence behavior in compressed sensing applications.

Experimental results

Research questions

• RQ1 Under what conditions does the method of averaged projections converge linearly for nonconvex feasibility problems?
• RQ2 Can local linear convergence be established when only one set is convex or prox-regular, rather than both?
• RQ3 How does the regularity modulus relate to the convergence rate in nonconvex settings?
• RQ4 Can inexact variants of the projection algorithms still achieve linear convergence?
• RQ5 Does the averaged projection method provide a constructive proof of the exact extremal principle?

Key findings

• The method of averaged projections converges locally at a linear rate when the intersection of closed sets is strongly regular, with the rate governed by a regularity modulus.
• Local linear convergence holds even if only one set is prox-regular or satisfies a constraint qualification, significantly broadening applicability.
• Inexact versions of the algorithms also converge linearly, making the results robust to computational errors.
• The convergence rate is Q-linear, meaning each iteration improves the error by a fixed factor, as confirmed numerically in a compressed sensing example.
• Numerical results show a convergence ratio of f(U_{k+1})/f(U_k) < 0.9627, indicating strong linear convergence in practice.
• The method provides a constructive algorithmic demonstration of the exact extremal principle in variational analysis.