[Paper Review] Compressed Counting Meets Compressed Sensing

Ping Li, Cun‐Hui Zhang+1 more|arXiv (Cornell University)|Oct 3, 2013

Sparse and Compressive Sensing Techniques21 references3 citations

TL;DR

This paper proposes a novel compressed sensing framework for nonnegative sparse signals using Compressed Counting (CC) based on maximally-skewed α-stable random projections. The method enables one-pass recovery with M = (Cα + o(1))ǫ⁻ᵅ(∑xᵢᵅ)log N/δ measurements, achieving ǫ-precision recovery in a single linear scan, and demonstrates robustness to measurement noise.

ABSTRACT

Compressed sensing (sparse signal recovery) has been a popular and important research topic in recent years. By observing that natural signals are often nonnegative, we propose a new framework for nonnegative signal recovery using Compressed Counting (CC). CC is a technique built on maximally-skewed p-stable random projections originally developed for data stream computations. Our recovery procedure is computationally very efficient in that it requires only one linear scan of the coordinates. Our analysis demonstrates that, when 00 and C=pi/2 when p=0.5. In particular, when p->0 the required number of measurements is essentially M=K\log N, where K is the number of nonzero coordinates of the signal.

Motivation & Objective

• To develop a fast, one-pass compressed sensing algorithm for nonnegative sparse signals.
• To leverage Compressed Counting (CC) with maximally-skewed α-stable distributions for efficient signal recovery.
• To establish theoretical sample complexity bounds for accurate recovery with additive ǫ-precision.
• To demonstrate robustness against measurement noise in practical settings.
• To enable application in high-speed data streams and resource-constrained environments.

Proposed method

• Uses maximally-skewed α-stable distributions (S(α,1,1)) for design matrix entries sᵢⱼ, enabling one-pass recovery.
• Employs the characteristic function E[exp(√−1Zλ)] = exp(−|λ|ᵅ(1−sign(λ)√−1 tan(πα/2))) for α ≠ 1.
• Applies the sampling procedure: sin(αu)/[sin u cos(απ/2)]^(1/α) × [sin(u−αu)/w]^{(1−α)/α} ∼ S(α,1,1), with w∼exp(1), u∼unif(0,π).
• Uses ratio statistics yⱼ/sᵢⱼ = xᵢ + noise terms, where noise is suppressed due to heavy-tailed S₁.
• Designs the measurement process to allow incremental updates in data streams: yⱼ^(t) = yⱼ^(t−1) + Iₜsᵢₜ,ⱼ.
• Leaves sparsified design matrices and coding of measurements as future work.

Experimental results

Research questions

• RQ1 Can maximally-skewed α-stable projections enable one-pass recovery of nonnegative sparse signals with provable accuracy?
• RQ2 What is the optimal sample complexity M for achieving ǫ-additive precision in nonnegative signal recovery?
• RQ3 How does the method perform under measurement noise compared to standard L1 solvers?
• RQ4 Can the framework be extended to signals with mixed positive and negative components?
• RQ5 What are the practical limits of α in terms of numerical stability and device constraints?

Key findings

• The method achieves ǫ-additive precision recovery with M = (Cα + o(1))ǫ⁻ᵅ(∑xᵢᵅ)log N/δ measurements, where Cα = 1 as α→0 and Cα = π/2 at α = 0.5.
• For α→0, the required measurements reduce to M ≈ K log N/δ, where K is the number of nonzero entries.
• The algorithm recovers all coordinates in a single linear scan, making it highly efficient for data streams.
• Experiments show robustness to measurement noise: CC remains accurate while L1 solvers (SPGL1, L1Magic) degrade significantly.
• Performance is not overly sensitive to α; good recovery is achieved even for α < 0.38 when M is set based on α→0.
• Theoretical analysis confirms sharp sample complexity bounds, and the method is robust due to the heavy-tailed nature of the stable distribution.