[Paper Review] Bias Reduction in Compressed Sensing

Carl Olsson, Marcus Carlsson+1 more|arXiv (Cornell University)|Dec 29, 2018

Sparse and Compressive Sensing TechniquesEngineering34 references3 citations

TL;DR

This paper proposes a novel bias-reduction framework for compressed sensing by combining non-convex sparsity and low-rank penalties with convex ℓ1- and nuclear-norm regularization via quadratic envelopes. The method achieves unbiased estimation with improved optimization stability, outperforming standard ℓ1 and nuclear norm relaxations in both synthetic and real-world computer vision tasks, particularly in non-rigid structure-from-motion and point cloud registration under noise and ill-posed conditions.

ABSTRACT

Sparsity and rank functions are important ways of regularizing under-determined linear systems. Optimization of the resulting formulations is made difficult since both these penalties are non-convex and discontinuous. The most common remedy is to instead use the $\ell^1$- and nuclear-norms. While these are convex and can therefore be reliably optimized they suffer from a shrinking bias that degrades the solution quality in the presence of noise. In this paper we combine recently developed bias free non-convex alternatives with the nuclear- and $\ell^1-$penalties. This reduces bias and still enables reliable optimization properties. We develop an efficient minimization scheme using derived proximal operators and evaluate the method on several real and synthetic computer vision applications with promising results.

Motivation & Objective

• . To address the shrinking bias inherent in ℓ1 and nuclear norm relaxations in compressed sensing.
• . To develop a non-convex, bias-free regularization framework that maintains global convergence guarantees under restricted isometry-like conditions.
• . To improve optimization stability in ill-posed problems by combining non-convex penalties with convex regularization terms.
• . To evaluate the method on real and synthetic computer vision tasks, including non-rigid structure from motion and point set registration.
• . To demonstrate that the proposed formulation achieves better reconstruction accuracy and robustness than standard ℓ1 and nuclear norm relaxations.

Proposed method

• . Uses the quadratic envelope Q2(f) to construct a non-convex relaxation of the combined penalty µ∥x∥0 + λ∥x∥1, ensuring better bias control than ℓ1 alone.
• . Applies the same principle to the rank and nuclear norm case: Q2(µrank(X) + λ∥X∥∗), enabling unbiased low-rank estimation.
• . Derives efficient proximal operators for the new penalty terms to enable scalable optimization using first-order methods.
• . Solves the resulting non-convex problem using an iterative minimization scheme with guaranteed convergence to the global minimizer under strong RLIP conditions.
• . Employs a hybrid regularization strategy: non-convex sparsity/rank penalty for bias reduction and convex ℓ1/nuclear norm for improved convergence basin.
• . Uses a fixed λ=5 in experiments to balance bias and stability, with µ varied to control sparsity and rank.

Experimental results

Research questions

• RQ1 . Can a hybrid regularization approach combining non-convex sparsity/rank penalties with convex ℓ1/nuclear norms reduce estimation bias in compressed sensing?
• RQ2 . Does the proposed method maintain global optimality guarantees under restricted isometry-like conditions while improving robustness to noise and ill-posedness?
• RQ3 . How does the inclusion of a weak ℓ1 or nuclear norm penalty affect the convergence basin of non-convex formulations in practice?
• RQ4 . In non-rigid structure-from-motion, does the proposed method yield better reconstruction accuracy than ℓ1 and nuclear norm relaxations, especially at high ranks?
• RQ5 . In point cloud registration, does the method produce fewer outliers and better inlier estimation than standard ℓ1 and nuclear norm formulations?

Key findings

• . The proposed method (48) consistently outperforms both ℓ1-norm (2) and pure non-convex rank penalty (47) in non-rigid structure-from-motion, achieving better data fit and lower distance to ground truth.
• . For low values of µ (high rank), the nuclear norm relaxation (49) suffers from shrinking bias, leading to larger reconstruction errors despite good data fit.
• . The hybrid formulation (48) maintains stable performance across a wide range of µ values, avoiding the instability seen in (47) at high ranks.
• . In point cloud registration, (14) produces fewer outliers and better inlier estimation than (5) and (2), especially in the presence of multiple strong hypotheses.
• . The method achieves superior performance in both synthetic and real-world experiments, demonstrating improved robustness and accuracy in ill-posed scenarios.
• . The addition of a weak λ∥X∥∗ term stabilizes the optimization process without introducing significant bias, enabling reliable recovery even when the RLIP condition is weak.