[Paper Review] Low-Complexity Iterative Algorithms for (Discrete) Compressed Sensing

Robert F. H. Fischer, Susanne Sparrer|arXiv (Cornell University)|May 19, 2017

Sparse and Compressive Sensing Techniques25 references3 citations

TL;DR

This paper proposes a low-complexity iterative compressed sensing algorithm using a Krylov subspace approximation to the optimal linear MMSE estimator, enabling efficient signal reconstruction with improved convergence over approximate message passing. The method unifies key components of turbo-like compressed sensing—estimation, unbiasing, and soft thresholding—while maintaining high performance for discrete signals across various distributions.

ABSTRACT

We consider iterative (`turbo') algorithms for compressed sensing. First, a unified exposition of the different approaches available in the literature is given, thereby enlightening the general principles and main differences. In particular we discuss i) the estimation step (matched filter vs. optimum MMSE estimator), ii) the unbiasing operation (implicitly or explicitly done and equivalent to the calculation of extrinsic information), and iii) thresholding vs. the calculation of soft values. Based on these insights we propose a low-complexity but well-performing variant utilizing a Krylov space approximation of the optimum linear MMSE estimator. The derivations are valid for any probability density of the signal vector. However, numerical results are shown for the discrete case. The novel algorithms shows very good performance and even slightly faster convergence compared to approximative message passing.

Motivation & Objective

• To address the high computational complexity of optimal MMSE estimation in iterative compressed sensing algorithms.
• To unify and clarify the core components of existing turbo-like compressed sensing approaches: estimation, unbiasing, and thresholding.
• To develop a practical, low-complexity alternative to approximate message passing (AMP) that maintains high reconstruction accuracy.
• To enable efficient signal recovery for discrete compressed sensing with minimal computational overhead.
• To generalize the framework to arbitrary signal probability distributions while demonstrating efficacy on discrete cases.

Proposed method

• Utilizes a Krylov subspace method to approximate the optimal linear MMSE estimator, reducing computational cost while preserving accuracy.
• Employs an iterative framework inspired by turbo processing, where extrinsic information is computed via unbiasing operations.
• Applies soft thresholding to the estimated signal values, with extrinsic information used to refine estimates across iterations.
• Integrates a matched filter or MMSE-based estimation step, with the proposed method favoring the MMSE approximation for improved performance.
• Explicitly handles unbiasing by computing extrinsic information, ensuring consistent information exchange between iterations.
• Supports any signal probability density function, with numerical validation focused on discrete signal models.

Experimental results

Research questions

• RQ1 How can the computational complexity of optimal MMSE estimation in iterative compressed sensing be reduced without sacrificing performance?
• RQ2 What is the impact of using a Krylov subspace approximation on convergence speed and reconstruction accuracy compared to standard AMP?
• RQ3 How do different estimation strategies (matched filter vs. MMSE) and thresholding methods affect the performance of iterative compressed sensing?
• RQ4 Can a unified framework be established to compare and contrast existing iterative compressed sensing algorithms in terms of estimation, unbiasing, and soft output computation?
• RQ5 To what extent does the proposed algorithm generalize across different signal distributions, particularly discrete ones?

Key findings

• The proposed algorithm achieves performance comparable to the optimal MMSE estimator while significantly reducing computational complexity.
• The Krylov subspace approximation enables faster convergence than conventional approximate message passing (AMP) in numerical experiments.
• The method maintains high reconstruction accuracy for discrete signals across various signal-to-noise ratios and sparsity levels.
• The use of extrinsic information through explicit unbiasing improves iterative convergence and estimation stability.
• The algorithm is robust across different signal distributions, with demonstrated effectiveness in the discrete case.
• The framework provides a unified view of existing iterative compressed sensing techniques, clarifying their underlying principles and differences.