[Paper Review] New explicit thresholding/shrinkage formulas for one class of regularization problems with overlapping group sparsity and their applications

Gang Liu, Ting‐Zhu Huang+2 more|arXiv (Cornell University)|Dec 24, 2013

Sparse and Compressive Sensing Techniques30 references3 citations

TL;DR

This paper proposes new explicit thresholding/shrinkage formulas for a class of overlapping group sparsity (OGS) regularization problems with translation-invariant overlapping groups, enabling efficient solution via alternating direction method of multipliers (ADMM). The key contribution is closed-form shrinkage solutions that avoid iterative majorization minimization, significantly improving computational efficiency while maintaining high accuracy in total variation-based image deblurring and denoising.

ABSTRACT

The least-square regression problems or inverse problems have been widely studied in many fields such as compressive sensing, signal processing, and image processing. To solve this kind of ill-posed problems, a regularization term (i.e., regularizer) should be introduced, under the assumption that the solutions have some specific properties, such as sparsity and group sparsity. Widely used regularizers include the $\ell_1$ norm, total variation (TV) semi-norm, and so on. Recently, a new regularization term with overlapping group sparsity has been considered. Majorization minimization iteration method or variable duplication methods are often applied to solve them. However, there have been no direct methods for solve the relevant problems because of the difficulty of overlapping. In this paper, we proposed new explicit shrinkage formulas for one class of these relevant problems, whose regularization terms have translation invariant overlapping groups. Moreover, we apply our results in TV deblurring and denoising with overlapping group sparsity. We use alternating direction method of multipliers to iterate solve it. Numerical results also verify the validity and effectiveness of our new explicit shrinkage formulas.

Motivation & Objective

• To address the computational inefficiency of existing methods for overlapping group sparsity (OGS) regularization problems, which rely on iterative majorization minimization or variable duplication.
• To develop closed-form thresholding/shrinkage formulas for a specific class of OGS regularization problems with translation-invariant overlapping groups.
• To apply these formulas to total variation (TV)-based image deblurring and denoising, improving both speed and solution quality.
• To establish convergence guarantees for the proposed ADMM-based algorithm under both L1 and L2 fidelity terms.
• To demonstrate the robustness and wide parameter range of the proposed method across different noise and blur levels.

Proposed method

• Proposes explicit shrinkage formulas for the optimization subproblems arising from OGS regularization with translation-invariant overlapping groups.
• Derives closed-form solutions for the soft-thresholding-like operations in the presence of overlapping group norms, avoiding inner-loop iterations.
• Applies the alternating direction method of multipliers (ADMM) to solve the overall optimization problem, leveraging the derived explicit formulas for fast updates.
• Extends the framework to both anisotropic (ATV) and isotropic (ITV) TV models for image deblurring and denoising.
• Uses weighted generalized ℓ2,1-norm regularization with identical weights across groups to maintain translation invariance and enable analytical tractability.
• Employs a relaxation parameter (γ = 1.618) in ADMM to accelerate convergence and stabilize iterations.

Experimental results

Research questions

• RQ1 Can explicit shrinkage formulas be derived for a class of overlapping group sparsity regularization problems with translation-invariant overlapping groups?
• RQ2 Can these formulas be efficiently integrated into an ADMM framework to solve image deblurring and denoising problems without iterative subproblem solvers?
• RQ3 How does the performance of the proposed method compare to existing methods in terms of PSNR, reconstruction error, and computational time?
• RQ4 Does the proposed method maintain robustness across varying noise levels and blur conditions?
• RQ5 Can the framework be extended to both anisotropic and isotropic TV models with overlapping group sparsity?

Key findings

• The proposed explicit shrinkage formulas eliminate the need for inner-loop iterative solvers, reducing computational cost per ADMM iteration to nearly the same level as traditional TV methods.
• For 30% salt-and-pepper noise, the proposed CATVOGSL1 and CITVOGSL1 methods achieve PSNR values of 32.30 dB and 32.08 dB respectively on image (d), outperforming Chan [6] and matching Liu [24] in PSNR while being faster.
• The method shows robustness in parameter selection, with consistent performance across different noise levels (30%, 40%, 50%) and image types.
• Numerical results show that the ATV and ITV variants of the proposed method achieve comparable or better PSNR and reconstruction error than Liu [24], which uses inner MM iterations.
• The ADMM-based algorithm with the new shrinkage formulas converges reliably, with convergence guarantees established for both L1 and L2 fidelity models.
• Visual results in Figure 6 confirm that the proposed methods preserve fine edges (e.g., handrails, windows) better than Chan [6], and are competitive with Liu [24] in edge recovery.