[Paper Review] Nonconvex set intersection problems: From projection methods to the Newton method for super-regular sets

C. H. Jeffrey Pang|arXiv (Cornell University)|Jun 27, 2015

Advanced Optimization Algorithms Research32 references3 citations

TL;DR

This paper extends the SHQP (supporting halfspace and quadratic programming) strategy from convex to nonconvex set intersection problems by leveraging super-regular sets—structured nonconvex sets with favorable variational properties. It proves linear convergence of the SHQP method under super-regularity and establishes superlinear and quadratic convergence when the problem geometry aligns with Newton-type methods, offering a practical acceleration framework for nonconvex feasibility problems.

ABSTRACT

The problem of finding a point in the intersection of closed sets can be solved by the method of alternating projections and its variants. It was shown in earlier papers that for convex sets, the strategy of using quadratic programming (QP) to project onto the intersection of supporting halfspaces generated earlier by the projection process can lead to an algorithm that converges multiple-term superlinearly. The main contributions of this paper are to show that this strategy can be effective for super-regular sets, which are structured nonconvex sets introduced by Lewis, Luke and Malick. Manifolds should be approximated by hyperplanes rather than halfspaces. We prove the linear convergence of this strategy, followed by proving that superlinear and quadratic convergence can be obtained when the problem is similar to the setting of the Newton method. We also show an algorithm that converges at an arbitrarily fast linear rate if halfspaces from older iterations are used to construct the QP.

Motivation & Objective

• To extend the SHQP strategy—previously effective for convex sets—to nonconvex settings using super-regular sets.
• To establish convergence guarantees (linear, superlinear, quadratic) for the SHQP method in nonconvex settings.
• To demonstrate that the method can achieve fast convergence by combining projection history with quadratic programming, even for nonconvex intersections.
• To provide algorithmic strategies that maintain efficiency by pruning outdated halfspaces or using warm-start techniques.

Proposed method

• The method uses supporting halfspaces generated by projections onto super-regular sets to define a polyhedral approximation of the intersection.
• At each iteration, a quadratic program is solved to project onto the intersection of these halfspaces, forming a new iterate.
• Older halfspaces are pruned to maintain numerical stability and computational efficiency, with warm-start techniques applied to the QP solver.
• The algorithm combines alternating projections with QP-based refinement, using a backtracking line search or active-set methods to ensure progress.
• Theoretical analysis leverages super-regularity to ensure that projections generate separating halfspaces near the solution.
• Global strategies include using averaged projections or avoiding regions near previous iterates to prevent stagnation.

Experimental results

Research questions

• RQ1 Can the SHQP strategy, effective for convex sets, be extended to nonconvex super-regular sets with provable convergence?
• RQ2 Under what conditions does the SHQP method achieve superlinear or quadratic convergence in nonconvex settings?
• RQ3 How can the number of supporting halfspaces be managed to ensure fast convergence without excessive computational cost?
• RQ4 What global strategies can be used to maintain convergence when the problem is nonconvex and the initial iterate is far from the solution?

Key findings

• The SHQP method converges linearly for super-regular sets under mild assumptions, generalizing convex convergence results.
• When the problem geometry resembles that of Newton’s method, the SHQP method achieves superlinear and even quadratic convergence.
• Arbitrarily fast linear convergence is possible if older halfspaces are retained in the QP subproblem.
• Pruning older halfspaces while using warm-start QP solvers maintains efficiency and avoids the Maratos effect.
• The method outperforms standard alternating projections in nonconvex settings, especially when the intersection is well-conditioned.
• Numerical examples show that even a small number of halfspaces can yield fast convergence, suggesting practical efficiency.