[Paper Review] Exploiting Correlation in Sparse Signal Recovery Problems: Multiple Measurement Vectors, Block Sparsity, and Time-Varying Sparsity

Zhilin Zhang, Bhaskar D. Rao|arXiv (Cornell University)|May 4, 2011

Sparse and Compressive Sensing Techniques16 references17 citations

TL;DR

This paper proposes T-SBL, a sparse Bayesian learning algorithm that adaptively learns and exploits correlation structures in multiple measurement vectors (MMV), block sparsity, and time-varying sparsity models. By modeling row-wise correlations in the solution matrix via a shared covariance matrix and using an iterative reweighted framework, T-SBL significantly improves reconstruction accuracy over existing methods, especially under high correlation, as validated in simulations with up to 90% failure rate reduction in correlated MMV settings.

ABSTRACT

A trend in compressed sensing (CS) is to exploit structure for improved reconstruction performance. In the basic CS model, exploiting the clustering structure among nonzero elements in the solution vector has drawn much attention, and many algorithms have been proposed. However, few algorithms explicitly consider correlation within a cluster. Meanwhile, in the multiple measurement vector (MMV) model correlation among multiple solution vectors is largely ignored. Although several recently developed algorithms consider the exploitation of the correlation, these algorithms need to know a priori the correlation structure, thus limiting their effectiveness in practical problems. Recently, we developed a sparse Bayesian learning (SBL) algorithm, namely T-SBL, and its variants, which adaptively learn the correlation structure and exploit such correlation information to significantly improve reconstruction performance. Here we establish their connections to other popular algorithms, such as the group Lasso, iterative reweighted $\ell_1$ and $\ell_2$ algorithms, and algorithms for time-varying sparsity. We also provide strategies to improve these existing algorithms.

Motivation & Objective

• To address the limitation of existing compressed sensing algorithms in exploiting correlation among multiple measurement vectors (MMV) and within sparse signal blocks.
• To develop a method that adaptively learns correlation structures without requiring prior knowledge of the correlation, unlike Kalman-filtered or fixed-structure approaches.
• To unify and improve existing iterative reweighted ℓ₁ and ℓ₂ algorithms for MMV and block-sparsity models by replacing ℓq norms with Mahalanobis distance using learned covariance matrices.
• To extend the applicability of sparse Bayesian learning to time-varying sparsity models by modeling them as concatenated MMV problems.

Proposed method

• T-SBL models the joint distribution of each row of the solution matrix X as a zero-mean multivariate normal with covariance γiB, where B is a shared, data-adaptive covariance matrix learned across rows.
• The algorithm uses a sparse Bayesian learning framework to iteratively update the solution vector x, the hyperparameters γi (indicating row sparsity), and the shared covariance matrix B using closed-form updates derived from marginal likelihood maximization.
• The method transforms the MMV model into a block-sparse form via Kronecker product and vectorization, enabling application of standard SBL techniques to the block-structured problem.
• T-SBL is shown to be equivalent to an iterative reweighted ℓ₂ and ℓ₁ algorithm, where the weights are updated based on the Mahalanobis distance using the learned covariance B, improving upon standard reweighting schemes.
• For time-varying sparsity, T-SBL is applied by segmenting the signal into multiple MMV blocks, each processed independently, with B estimated per block to capture temporal correlation.
• The algorithm is validated through simulations comparing T-SBL and its simplified variant T-MSBL against state-of-the-art methods in MMV and time-varying sparsity settings.

Experimental results

Research questions

• RQ1 Can a sparse Bayesian learning framework effectively learn and exploit unknown correlation structures in multiple measurement vectors without prior knowledge of the correlation?
• RQ2 How does replacing ℓq norms with Mahalanobis distance in iterative reweighted ℓ₁ and ℓ₂ algorithms improve performance in block-sparse and MMV recovery?
• RQ3 To what extent can T-SBL improve reconstruction performance in time-varying sparsity models compared to existing Kalman-filtered and least-squares compressed sensing methods?
• RQ4 What is the impact of adaptive covariance estimation on failure rates in correlated MMV settings?

Key findings

• In the MMV model with high correlation (ρ=0.9), T-SBL and T-MSBL reduced failure rates to near zero, while other algorithms failed in over 90% of trials.
• When correlation was absent (ρ=0), T-SBL and T-MSBL maintained high performance, demonstrating robustness across correlation levels.
• Replacing ℓ₂ norms with Mahalanobis distance in iterative reweighted ℓ₁ algorithms led to significant performance gains, especially when the true correlation structure was known or estimated.
• In time-varying sparsity experiments, T-SBL outperformed both KF-CS and LS-CS, and also surpassed M-SBL, particularly in scenarios with high temporal correlation (ρ=0.99).
• The use of a shared covariance matrix B across rows in T-SBL prevented overfitting and enabled effective correlation learning, even in noisy or low-SNR conditions.
• T-SBL’s performance was robust to noise, as it jointly estimated the noise variance and correlation structure, outperforming methods requiring known noise parameters.