[Paper Review] Algorithms and software for projections onto intersections of convex and non-convex sets with applications to inverse problems

Bas Peters, Felix J. Herrmann|arXiv (Cornell University)|Feb 26, 2019

Sparse and Compressive Sensing Techniques87 references4 citations

TL;DR

This paper proposes PARSDMM, a novel algorithm and software package called SetIntersectionProjection for efficiently computing projections onto intersections of multiple convex and non-convex constraint sets in inverse problems. By leveraging adaptive penalty parameter selection, parallel computation, and multilevel acceleration, the method outperforms Dykstra’s algorithm and enables effective regularization using diverse prior knowledge without trade-off parameters.

ABSTRACT

We propose algorithms and software for computing projections onto the intersection of multiple convex and non-convex constraint sets. The software package, called SetIntersectionProjection, is intended for the regularization of inverse problems in physical parameter estimation and image processing. The primary design criterion is working with multiple sets, which allows us to solve inverse problems with multiple pieces of prior knowledge. Our algorithms outperform the well known Dykstra's algorithm when individual sets are not easy to project onto because we exploit similarities between constraint sets. Other design choices that make the software fast and practical to use, include recently developed automatic selection methods for auxiliary algorithm parameters, fine and coarse grained parallelism, and a multilevel acceleration scheme. We provide implementation details and examples that show how the software can be used to regularize inverse problems. Results show that we benefit from working with all available prior information and are not limited to one or two regularizers because of algorithmic, computational, or hyper-parameter selection issues.

Motivation & Objective

• Address the computational challenge of projecting onto intersections of multiple convex and non-convex constraint sets in inverse problems.
• Overcome limitations of existing methods like Dykstra’s algorithm, which struggle when individual sets are difficult to project onto.
• Enable the use of multiple, diverse pieces of prior knowledge (e.g., bounds, sparsity, low-rank, total variation) in regularization without introducing trade-off parameters.
• Develop a practical, high-performance software package that supports parallelism and automatic parameter selection for real-world inverse problems.
• Provide a robust stopping criterion based on feasibility and relative evolution of iterates to ensure convergence without excessive computation.

Proposed method

• Propose the Projection Adaptive Relaxed Simultaneous Direction Method of Multipliers (PARSDMM), a variant of the ADMM framework tailored for set intersection projections.
• Introduce adaptive update rules for penalty parameters (ρ) and relaxation parameters (γ) using local dual variable and residual information to improve convergence speed.
• Decompose the projection problem into parallel subproblems via linear transformations (A_i), enabling fine- and coarse-grained parallelism.
• Use a multilevel acceleration scheme that exploits structural similarities between constraint sets to reduce computational cost.
• Implement a dual stopping criterion combining normalized transform-domain feasibility error (r^feas_i) and relative evolution of iterates (r^evol) to detect convergence robustly.
• Integrate efficient projection operators for common sets such as norm-balls, bound constraints, cardinality, and low-rank matrices.

Experimental results

Research questions

• RQ1 Can we efficiently compute projections onto the intersection of multiple convex and non-convex sets when individual projections are computationally expensive?
• RQ2 How can we design an adaptive parameter selection strategy for ADMM-based solvers that improves convergence without manual tuning?
• RQ3 To what extent does exploiting structural similarities between constraint sets improve computational performance in large-scale inverse problems?
• RQ4 Can the proposed method outperform established algorithms like Dykstra’s in terms of speed and robustness while handling complex, non-convex constraints?
• RQ5 How effective is the proposed stopping criterion in balancing solution accuracy and computational cost in practical inverse problem applications?

Key findings

• The PARSDMM algorithm significantly outperforms Dykstra’s algorithm in convergence speed and robustness, especially when projecting onto complex or non-convex sets.
• The use of adaptive penalty and relaxation parameters reduces the need for manual tuning and accelerates convergence across diverse inverse problem types.
• Parallelization across constraint sets enables substantial speedups, particularly on multi-core or GPU-accelerated architectures.
• The multilevel acceleration scheme effectively reduces computational cost by exploiting similarities between constraint sets, such as shared transform domains.
• The dual stopping criterion—based on feasibility and relative evolution—ensures accurate convergence detection with minimal overhead, achieving ε^evol = 10⁻² and ε^feas_i = 10⁻³ in numerical experiments.
• The software package SetIntersectionProjection successfully regularizes challenging inverse problems such as joint denoising-deblurring-inpainting and image desaturation by incorporating multiple prior constraints without trade-off parameters.