[Paper Review] Improved analysis of algorithms based on supporting halfspaces and quadratic programming for the convex intersection and feasibility problems

C. H. Jeffrey Pang|arXiv (Cornell University)|Jun 16, 2014

Optimization and Variational Analysis27 references3 citations

TL;DR

This paper enhances algorithms for convex feasibility and intersection problems by integrating supporting halfspaces and quadratic programming (QP), achieving multiple-term superlinear or quadratic convergence under new regularity conditions—specifically, the second-order supporting hyperplane property (SOSH). It enables finite infeasibility detection and uses the dual active-set QP algorithm of Goldfarb and Idnani to obtain useful iterates via partial QP solves, avoiding full solution of large QPs.

ABSTRACT

This paper improves the algorithms based on supporting halfspaces and quadratic programming for convex set intersection problems in our earlier paper in several directions. First, we give conditions so that much smaller quadratic programs (QPs) and approximate projections arising from partially solving the QPs are sufficient for multiple-term superlinear convergence for nonsmooth problems. Second, we identify additional regularity, which we call the second order supporting hyperplane property (SOSH), that gives multiple-term quadratic convergence. Third, we show that these fast convergence results carry over for the convex inequality problem. Fourth, we show that infeasibility can be detected in finitely many operations. Lastly, we explain how we can use the dual active set QP algorithm of Goldfarb and Idnani to get useful iterates by solving the QPs partially, overcoming the problem of solving large QPs in our algorithms.

Motivation & Objective

• Address slow convergence in convex feasibility and intersection problems (SIP) and convex inequality problems (CIP) using accelerated projection methods.
• Overcome the computational burden of solving large quadratic programs (QPs) to optimality in existing algorithms.
• Establish conditions under which smaller QPs and approximate projections yield fast convergence, even for nonsmooth problems.
• Provide finite convergence detection for infeasible instances of SIP and CIP.
• Enable practical implementation using the dual active-set QP algorithm of Goldfarb and Idnani with warm-starting and partial solves.

Proposed method

• Construct outer approximations of the intersection set K using supporting halfspaces generated from projections onto individual convex sets.
• Project iterates onto the intersection of these halfspaces via quadratic programming (QP), using the identity matrix as the QP Hessian for well-posedness.
• Introduce the second-order supporting hyperplane property (SOSH) as a regularity condition enabling multiple-term quadratic convergence.
• Apply the dual active-set QP algorithm of Goldfarb and Idnani (GI algorithm) to solve QPs partially, maintaining feasibility and improving iterates incrementally.
• Use inner GI steps as a subroutine to generate better iterates for the SIP and CIP without full QP solution, leveraging monotonic decrease in distance to the solution set.
• Leverage warm-starting in the GI algorithm to accelerate QP solves across iterations, improving efficiency in sequential applications.

Experimental results

Research questions

• RQ1 Under what conditions can smaller quadratic programs and approximate projections still yield multiple-term superlinear convergence in nonsmooth convex feasibility problems?
• RQ2 What regularity conditions, beyond standard constraint qualifications, ensure multiple-term quadratic convergence in the SIP and CIP?
• RQ3 Can infeasibility in the SIP and CIP be detected in finitely many operations, and under what assumptions?
• RQ4 How can the dual active-set QP algorithm be used effectively to generate useful intermediate iterates when full QP solutions are computationally prohibitive?
• RQ5 To what extent can the convergence speed of alternating projection-type methods be accelerated by combining supporting halfspaces with QP-based projections?

Key findings

• The paper establishes that under the second-order supporting hyperplane property (SOSH), multiple-term quadratic convergence is achieved for the SIP and CIP, even in nonsmooth settings.
• Finite convergence to infeasibility is guaranteed under reasonable regularity conditions, allowing a certificate of infeasibility to be generated in finitely many operations.
• Partial solutions of quadratic programs using the Goldfarb–Idnani dual active-set algorithm yield iterates that are strictly better than the current iterate for the feasibility problem, even before optimality is reached.
• The sequence of iterates generated by the GI algorithm monotonically improves the distance to the solution set, with the error decreasing by at least the norm of the step taken in each inner iteration.
• The use of inner GI steps allows for warm-starting of QP solves across outer iterations, significantly reducing computational cost in practice.
• The proposed method achieves superlinear or quadratic convergence rates in the SIP and CIP by leveraging polyhedral outer approximations and QP-based projections, outperforming standard alternating projections.