[Paper Review] Fast L1-L2 minimization via a proximal operator

Yifei Lou, Ming Yan|arXiv (Cornell University)|Sep 29, 2016

Sparse and Compressive Sensing Techniques28 references3 citations

TL;DR

This paper proposes a fast algorithm for L1-L2 minimization in compressive sensing by deriving an analytical proximal operator for the L1-L2 functional, enabling efficient use of forward-backward splitting (FBS) and ADMM. The method achieves significantly faster convergence than the traditional difference-of-convex algorithm (DCA), with up to 5× speedup in numerical experiments, while maintaining superior recovery performance for coherent sensing matrices.

ABSTRACT

This paper aims to develop new and fast algorithms for recovering a sparse vector from a small number of measurements, which is a fundamental problem in the field of compressive sensing (CS). Currently, CS favors incoherent systems, in which any two measurements are as little correlated as possible. In reality, however, many problems are coherent, and conventional methods such as $L_1$ minimization do not work well. Recently, the difference of the $L_1$ and $L_2$ norms, denoted as $L_1$-$L_2$, is shown to have superior performance over the classic $L_1$ method, but it is computationally expensive. We derive an analytical solution for the proximal operator of the $L_1$-$L_2$ metric, and it makes some fast $L_1$ solvers such as forward-backward splitting (FBS) and alternating direction method of multipliers (ADMM) applicable for $L_1$-$L_2$. We describe in details how to incorporate the proximal operator into FBS and ADMM and show that the resulting algorithms are convergent under mild conditions. Both algorithms are shown to be much more efficient than the original implementation of $L_1$-$L_2$ based on a difference-of-convex approach in the numerical experiments.

Motivation & Objective

• Address the computational inefficiency of existing L1-L2 minimization methods based on the difference-of-convex algorithm (DCA), which is slow despite strong recovery performance.
• Develop a fast, convergent algorithm for L1-L2 minimization by deriving an analytical proximal operator for the nonconvex L1-L2 functional.
• Enable the application of efficient first-order solvers like FBS and ADMM to L1-L2 minimization by leveraging the derived proximal operator.
• Demonstrate that the proposed method outperforms both standard L1 minimization and nonconvex Lp (p<1) methods in terms of speed and robustness for highly coherent sensing matrices.
• Investigate the use of continuation strategies in the regularization parameter α to further improve recovery quality and convergence behavior.

Proposed method

• Derive an analytical closed-form solution for the proximal operator of the L1-L2 functional, rα(x) = ||x||₁ - α||x||₂, which generalizes soft thresholding for L1 minimization.
• Integrate the derived proximal operator into the forward-backward splitting (FBS) and alternating direction method of multipliers (ADMM) frameworks to solve the unconstrained minimization problem E(x) = rα(x) + l(x), where l(x) has a Lipschitz continuous gradient.
• Prove global convergence of the FBS and ADMM variants under mild conditions, ensuring stability and reliability of the proposed algorithms.
• Implement adaptive parameter updating for the penalty parameter γ in FBS using a sigmoid-based rule to improve convergence speed and robustness.
• Compare the performance of the new FBS and ADMM solvers with the standard DCA approach and other nonconvex sparsity-promoting methods like L1/2 via half-thresholding.
• Use numerical experiments on both noiseless and noisy compressive sensing problems to validate the efficiency and accuracy of the proposed algorithms.

Experimental results

Research questions

• RQ1 Can an analytical proximal operator be derived for the nonconvex L1-L2 functional to enable efficient optimization with standard first-order methods?
• RQ2 How does the performance of FBS and ADMM with the new proximal operator compare to the traditional DCA approach in terms of convergence speed and solution quality?
• RQ3 Does the proposed method outperform L1 minimization and other nonconvex sparsity-inducing penalties (e.g., L1/2) in recovering sparse signals from coherent sensing matrices?
• RQ4 What is the impact of adaptive parameter selection (e.g., γ) on the convergence and accuracy of the FBS-based L1-L2 solver?
• RQ5 Can a continuation strategy in the parameter α improve the recovery performance of the L1-L2 method for highly coherent sensing matrices?

Key findings

• The proposed method achieves up to 5× speedup over the DCA-based implementation of L1-L2 minimization, with computational times reduced from ~3.36 seconds (DCA) to ~0.31 seconds (ADMM) for a 100×1500 problem.
• For coherent sensing matrices (e.g., DCT, F=5), the L1-L2 method with FBS and ADMM outperforms L1/2 minimization via half-thresholding in terms of recovery accuracy when the number of measurements M is small.
• In noisy settings with M=238, the L1-L2 method (FBS) achieved an MSE of 5.71 (±0.79), outperforming L1 (MSE: 5.83) and L1/2 (MSE: 6.91), while being significantly faster than L1/2 (1.92 sec vs. 0.57 sec).
• ADMM with fixed γ=0.8 achieved the best balance of speed and accuracy, with computational time of 0.31 seconds and MSE of 5.09 (±0.65) at M=250, outperforming DCA and L1/2 in both metrics.
• The DCA approach yielded better recovery results than ADMM for coherent matrices, motivating the use of a continuation strategy in α to improve performance.
• The adaptive γ update rule in FBS improved convergence and robustness, with consistent performance across different problem sizes and noise levels.