[Paper Review] A generalization of some random variables involving in certain compressive sensing problems

Romeo Meštrović|arXiv (Cornell University)|Jun 26, 2018

Sparse and Compressive Sensing Techniques17 references4 citations

TL;DR

This paper generalizes a discrete complex-valued random variable used in compressive sensing, deriving exact expressions for its expected value and variance when sampling m entries from an N-element multiset. The key contribution is linking these moments to matrix properties—Frobenius norm, coherence, singular values, and the Welch bound—providing theoretical tools for analyzing measurement matrices in compressive sensing.

ABSTRACT

In this paper we give a generalization of the discrete complex-valued random variable defined and investigated in \cite{ssa} and \cite{m8}. We prove the statements concerning the expressions for the excepted value and the variance of this random variable. In partucular, such a random variable here is defined for each of $m$ rows of any $m imes N$ complex or real matrix ${ m{\bf A}}$ with $1\le m\le N$. We consider the arithmetic mean $\bar{X}(m)$ of these $m$ random variables and we deduce the expressions for the expected value $\Bbb E[\bar{X}(m)]$ and the variance ${ m{ Var}}[\bar{X}(m)]$ of $\bar{X}(m)$. Using the expression for ${ m{ Var}}[\bar{X}(m)]$, we establish some equalities and inequalities involving ${ m{Var}}[\bar{X}(m)]$, the Frobenius norm, the largest eigenvalue, the largest singular value and the coherence of a matrix ${ m{\bf A}}$. It is showed that some of these estimates are closely related to the Welch bound of the coherence of a $m imes N$ complex or real matrix ${ m{\bf A}}$ with $1\le m\le N$. Taking into account that the value of coherence of the measurement matrix in the theory of compressive sensing has a significant role, we believe that our results should be useful for some topics of this theory.

Motivation & Objective

• To generalize a discrete complex-valued random variable used in compressive sensing for arbitrary m-row submatrices of an m×N matrix.
• To derive exact analytical expressions for the expected value and variance of the sum of m randomly selected entries from a multiset of complex numbers.
• To relate the variance of the sample mean to fundamental matrix norms and coherence, particularly connecting to the Welch bound.
• To provide theoretical tools for assessing the performance of measurement matrices in compressive sensing via probabilistic moments.

Proposed method

• Define a discrete complex-valued random variable X(m,Φ_N) as the sum of m uniformly random, without-replacement selected elements from a multiset Φ_N of N complex numbers.
• Use combinatorial counting to derive the expected value E[X(m,Φ_N)] = (m/N)∑z_i, leveraging symmetry and hypergeometric counting.
• Compute the second moment E[|X(m,Φ_N)|²] by counting occurrences of |z_i|² and z_i z̄_j terms across all m-subsets, applying binomial identities.
• Derive the variance Var[X(m,Φ_N)] using the identity Var[X] = E[|X|²] − |E[X]|², yielding a closed-form expression in terms of ∑|z_i|² and |∑z_i|².
• Introduce the normalized sample mean X̄(m,Φ_N) = X(m,Φ_N)/m and derive its expected value and variance to analyze mean stability.
• Relate the variance expression to matrix norms: Frobenius norm, largest singular value, eigenvalues, and coherence, especially connecting to the Welch bound.

Experimental results

Research questions

• RQ1 How can the expected value and variance of the sum of m randomly selected entries from a complex multiset be expressed in closed form?
• RQ2 What is the relationship between the variance of the sample mean and the coherence of a measurement matrix in compressive sensing?
• RQ3 How do the derived expressions for variance relate to known bounds such as the Welch bound on matrix coherence?
• RQ4 Can the variance of the sample mean be expressed in terms of fundamental matrix norms like the Frobenius norm and spectral properties?
• RQ5 What is the role of the multiset structure and sampling without replacement in shaping the distributional properties of the sum in compressive sensing contexts?

Key findings

• The expected value of the sum X(m,Φ_N) is exactly E[X(m,Φ_N)] = (m/N)∑_{i=1}^N z_i, showing linear scaling with the sample size m.
• The variance of X(m,Φ_N) is given by Var[X(m,Φ_N)] = [m(N−m)/(N²(N−1))]·(N∑|z_i|² − |∑z_i|²), which depends on the spread of the entries.
• The variance can also be expressed as Var[X(m,Φ_N)] = [m(N−m)/(N²(N−1))]·∑_{1≤i<k≤N} |z_i − z_k|², linking it to pairwise differences.
• For the normalized sample mean X̄(m,Φ_N), the variance is Var[X̄(m,Φ_N)] = [(N−m)/(mN²(N−1))]·(N∑|z_i|² − |∑z_i|²), showing decreasing variance with larger m.
• The derived variance expression is directly related to the coherence of the matrix A, with equality conditions tied to the Welch bound.
• The results establish inequalities and equalities connecting the variance of the sample mean to the Frobenius norm, largest singular value, and coherence of the matrix A, providing theoretical bounds for compressive sensing applications.