[Paper Review] Alternating conditional gradient method for convex feasibility problems

R. Díaz Millán, O. P. Ferreira+1 more|arXiv (Cornell University)|Dec 9, 2019

Optimization and Variational Analysis28 references4 citations

TL;DR

This paper proposes the Alternating Conditional Gradient (ACondG) method for convex feasibility problems, combining exact projections on one set with inexact projections via the conditional gradient (CondG) method on another. It ensures convergence to a point in the intersection if nonempty, or to the closest points between disjoint sets, with faster convergence and fewer iterations than exact projection methods in numerical tests.

ABSTRACT

The classical convex feasibility problem in a finite dimensional Euclidean space is studied in the present paper. We are interested in two cases. First, we assume to know how to compute an exact project onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume that both sets involved are compact such that the CondG method can be used for computing efficiently inexact projections on them. We combine alternating projection method with CondG method to design a new method, which can be seen as an inexact feasible version of alternate projection method. The proposed method generates two different sequences belonging to each involved set, which converge to a point in the intersection of them whenever it is not empty. If the intersection is empty, then the sequences converge to points in the respective sets whose distance is equal to the distance between the sets in consideration.

Motivation & Objective

• To address convex feasibility problems in finite-dimensional Euclidean spaces where exact projections are computationally expensive.
• To develop a method that uses inexact projections via the conditional gradient (CondG) method to reduce computational cost per iteration.
• To ensure convergence to a feasible point in the intersection of two convex sets when it exists, or to the closest points between disjoint sets.
• To demonstrate the superiority of inexact projections via CondG over exact projections in terms of iteration count and robustness in numerical experiments.

Proposed method

• The method alternates between projecting onto one set using exact projection and onto the other using the conditional gradient (CondG) method for inexact, efficient projections.
• It generates two sequences, $ (x^k) \subset A $ and $ (y^k) \subset B $, which converge to a point in $ A \cap B $ if the intersection is nonempty.
• For disjoint sets, the sequences converge to points $ x^* \in A $, $ y^* \in B $ such that $ \|x^* - y^* =\mathrm{dist}(A,B) $.
• The CondG method is used to compute inexact projections on compact sets with special structures, exploiting sparsity and separability for low per-iteration cost.
• The algorithm is designed as an inexact variant of the classical alternating projection method, with increasing accuracy of inexact projections over iterations.
• Two versions are proposed: one with exact projection on one set and CondG on a compact set, and another with CondG on both compact sets.

Experimental results

Research questions

• RQ1 Can the alternating projection method be made more efficient by replacing exact projections with inexact ones computed via the conditional gradient method?
• RQ2 Does the proposed ACondG method converge to a feasible point in the intersection of two convex sets when the intersection is nonempty?
• RQ3 What happens to the iterates when the intersection is empty—does the method converge to the closest points between the sets?
• RQ4 How does the performance of the inexact ACondG method compare to the exact alternating projection method in terms of iteration count and feasibility violation?
• RQ5 Can the use of CondG for inexact projections lead to faster convergence, especially in problems with interior intersections?

Key findings

• In feasible instances with nonempty interior intersections, ACondG-2 found a feasible point in 2, 2, 8, and 155 iterations, respectively, while the exact method failed to progress.
• For infeasible instances, ACondG-2 achieved feasibility violation measures of $ \mathcal{O}(10^{-4}) $, $ \mathcal{O}(10^{-3}) $, $ \mathcal{O}(10^{-2}) $, and $ \mathcal{O}(10^{-1}) $ in 33, 5, 3, and 2 iterations, respectively.
• The exact method required 82, 15, 5, and 2 iterations to reach similar feasibility violations, indicating ACondG-2 converges faster to the nearest region between sets.
• On average, ACondG-2 required 65.7% fewer iterations than the exact method before stopping due to lack of progress.
• The method demonstrated robustness and faster convergence, particularly in problems with interior intersections, where convergence occurred in a finite number of iterations.
• Numerical results suggest that inexact projections via CondG outperform exact projections in terms of iteration efficiency and robustness in the tested class of problems.