[Paper Review] Statistical Mechanical Analysis of Compressed Sensing Utilizing Correlated Compression Matrix

Koujin Takeda, Yoshiyuki Kabashima|arXiv (Cornell University)|Jan 25, 2010

Sparse and Compressive Sensing Techniques7 references4 citations

TL;DR

This paper analyzes the reconstruction limit of compressed sensing using an $L_1$-norm minimization scheme with a Kronecker-type correlated compression matrix via statistical mechanics. It finds that strong one-dimensional correlations between expansion bases slightly degrade reconstruction performance, increasing the critical compression rate $\alpha_c$ by approximately 1% compared to the uncorrelated case.

ABSTRACT

We investigate a reconstruction limit of compressed sensing for a reconstruction scheme based on the L1-norm minimization utilizing a correlated compression matrix with a statistical mechanics method. We focus on the compression matrix modeled as the Kronecker-type random matrix studied in research on multi-input multi-output wireless communication systems. We found that strong one-dimensional correlations between expansion bases of original information slightly degrade reconstruction performance.

Motivation & Objective

• To evaluate the reconstruction limit $\alpha_c$ of $L_1$-norm compressed sensing when the compression matrix exhibits correlations.
• To extend prior i.i.d. matrix analyses to structured, correlated compression matrices inspired by MIMO wireless systems.
• To investigate how correlations in the expansion bases (via $\bm{R}_{\mathrm{t}}$) affect reconstruction performance.
• To validate the replica method results through numerical experiments and extrapolation.

Proposed method

• Employs the replica method from statistical mechanics to compute the quenched free energy and derive the cost function for $L_p$-norm reconstruction.
• Models the compression matrix as $\bm{F} = \sqrt{\bm{R}_{\mathrm{r}}}\bm{\Xi}\sqrt{\bm{R}_{\mathrm{t}}}$, where $\bm{\Xi}$ is i.i.d. Gaussian and $\bm{R}_{\mathrm{t}}, \bm{R}_{\mathrm{r}}$ are correlation matrices.
• Performs asymptotic analysis in the limit $N, P \to \infty$ with fixed $\alpha = P/N$, focusing on the $L_1$-norm case.
• Uses numerical integration over 100 samples to evaluate the multiple integrals in the cost function expression.
• Validates results via convex optimization experiments in MATLAB and extrapolates $\alpha_c$ to the $N \to \infty$ limit using quadratic regression.

Experimental results

Research questions

• RQ1 How does correlation in the expansion bases of the original signal affect the $L_1$-norm reconstruction limit in compressed sensing?
• RQ2 What is the critical compression rate $\alpha_c$ for a Kronecker-type correlated compression matrix?
• RQ3 Does the correlation in the observation vectors ($\bm{R}_{\mathrm{r}}$) significantly impact reconstruction performance?
• RQ4 How does the reconstruction limit compare between correlated and uncorrelated compression matrices?

Key findings

• The reconstruction limit $\alpha_c$ increases by approximately 1% when the correlation parameter $r$ in the tridiagonal $\bm{R}_{\mathrm{t}}$ reaches 0.5, indicating degraded performance.
• For $\rho = 0.5$, the critical compression rate increases from $\alpha_c \approx 0.8312$ (uncorrelated case) to $\alpha_c = 0.84057(14)$ under strong correlation ($r=0.5$).
• The replica analysis result $\alpha_c = 0.84057(14)$ is validated by numerical experiments, which extrapolate to $\alpha_c = 0.84017(28)$ for $N \to \infty$.
• The correlation matrix $\bm{R}_{\mathrm{r}}$ describing observation vector correlations does not affect the reconstruction limit, as it can be absorbed via redefinition of $\bm{\Xi}$.
• The $L_1$-norm reconstruction is robust to weak correlations but shows measurable performance degradation under strong one-dimensional correlations in the expansion bases.
• The analysis framework can be reinterpreted to describe correlated original coefficients with uncorrelated bases, preserving the same critical rate.