[Paper Review] Multiple Measurement Vectors Problem: A Decoupling Property and its Applications

Saeid Haghighatshoar, Giuseppe Caire|arXiv (Cornell University)|Oct 31, 2018

Sparse and Compressive Sensing Techniques22 references4 citations

TL;DR

This paper establishes a decoupling property for ℓ₂,₁-regularized least squares (ℓ₂,₁-LS) in the Multiple Measurement Vectors (MMV) problem, proving that the algorithm separates into a covariance estimation phase followed by individual signal reconstruction via MMSE. This decomposition enables precise analysis of how signal correlations and dictionary mismatch affect performance, and leads to improved MMV algorithms beyond ℓ₂,₁-LS.

ABSTRACT

We study a Compressed Sensing (CS) problem known as Multiple Measurement Vectors (MMV) problem, which arises in joint estimation of multiple signal realizations when the signal samples have a common (joint) sparse support over a fixed known dictionary. Although there is a vast literature on the analysis of MMV, it is not yet fully known how the number of signal samples and their statistical correlations affects the performance of the joint estimation in MMV. Moreover, in many instances of MMV the underlying sparsifying dictionary may not be precisely known, and it is still an open problem to quantify how the dictionary mismatch may affect the estimation performance. In this paper, we focus on $\ell_{2,1}$-norm regularized least squares ($\ell_{2,1}$-LS) as a well-known and widely-used MMV algorithm in the literature. We prove an interesting decoupling property for $\ell_{2,1}$-LS, where we show that it can be decomposed into two phases: i) use all the signal samples to estimate the signal covariance matrix (coupled phase), ii) plug in the resulting covariance estimate as the true covariance matrix into the Minimum Mean Squared Error (MMSE) estimator to reconstruct each signal sample individually (decoupled phase). As a consequence of this decomposition, we are able to provide further insights on the performance of $\ell_{2,1}$-LS for MMV. In particular, we address how the signal correlations and dictionary mismatch affects its performance. Moreover, we show that by using the decoupling property one can obtain a variety of MMV algorithms with performances even better than that of $\ell_{2,1}$-LS. We also provide numerical simulations to validate our theoretical results.

Motivation & Objective

• To understand how the number of signal samples and their statistical correlations affect joint estimation performance in the Multiple Measurement Vectors (MMV) problem.
• To quantify the impact of dictionary mismatch on the performance of ℓ₂,₁-LS in MMV when the sparsifying dictionary is not perfectly known.
• To establish a theoretical foundation for analyzing ℓ₂,₁-LS performance through a novel decoupling decomposition.
• To leverage the decoupling property to design improved MMV algorithms with better reconstruction accuracy than ℓ₂,₁-LS.

Proposed method

• Prove that ℓ₂,₁-LS can be decomposed into two phases: (i) joint estimation of the signal covariance matrix using all signal samples, and (ii) individual signal reconstruction via Minimum Mean Squared Error (MMSE) using the estimated covariance.
• Use matrix inversion and determinant identities (e.g., rank-1 update) to derive closed-form expressions for the derivative of the objective function with respect to the dual variable vector γ.
• Apply a coordinate-wise steepest descent algorithm to optimize the dual variable vector γ, using bisection to solve for the optimal step size in each coordinate update.
• Derive the derivative of the cost function gₖ(d) with respect to the dual variable increment d, and identify the largest root of g′ₖ(d) = 0 as the optimal update direction.
• Enforce non-negativity constraints on the dual variables by setting d* = max{d₀, -γₖ}, where d₀ is the largest solution to g′ₖ(d) = 0.
• Use the decoupled structure to analyze performance under varying signal correlation and dictionary mismatch, and to design enhanced MMV algorithms.

Experimental results

Research questions

• RQ1 How does the number of signal samples and their statistical correlation influence the performance of ℓ₂,₁-LS in the MMV problem?
• RQ2 What is the effect of dictionary mismatch on the estimation accuracy of ℓ₂,₁-LS when the true sparsifying dictionary is not perfectly known?
• RQ3 Can the ℓ₂,₁-LS algorithm be decomposed into a two-phase process that separates joint covariance estimation from individual signal reconstruction?
• RQ4 Does the decoupling property enable the design of MMV algorithms with better performance than ℓ₂,₁-LS?
• RQ5 How can the dual optimization problem for ℓ₂,₁-LS be efficiently solved using coordinate-wise descent with bisection-based step size selection?

Key findings

• The ℓ₂,₁-LS estimator for MMV admits a decoupling property: it can be decomposed into a coupled phase estimating the signal covariance matrix and a decoupled phase using MMSE with the estimated covariance.
• The decoupling property enables precise theoretical analysis of how signal correlation and dictionary mismatch degrade or improve estimation performance.
• The optimal dual variable update in the coordinate-wise descent algorithm is found by solving g′ₖ(d) = 0 using bisection, with the largest root selected as the minimizer.
• The method ensures non-negativity of the dual variables by setting d* = max{d₀, -γₖ}, where d₀ is the largest solution to the derivative equation.
• The derived optimization framework allows for the design of new MMV algorithms that outperform ℓ₂,₁-LS in terms of mean squared error.
• Numerical simulations validate the theoretical findings, confirming the accuracy of the decoupling approximation and the effectiveness of the proposed algorithmic framework.