[Paper Review] Geometric approach to error correcting codes and reconstruction of signals

Mark Rudelson, Roman Vershynin|arXiv (Cornell University)|Feb 15, 2005

Medical Imaging Techniques and ApplicationsMedicine28 references19 citations

TL;DR

This paper introduces a geometric functional analysis approach to error-correcting codes and signal reconstruction using the $ι_1$-norm minimization (Basis Pursuit) as a decoder. It proves that for most random linear transforms $Q: \mathbb{R}^n \to \mathbb{R}^m$, signals with small support or corrupted components can be exactly reconstructed when $m \gtrsim r \log(n/r)$, establishing a sharp threshold for robust recovery via $\ell_1$-projection.

ABSTRACT

We develop an approach through geometric functional analysis to error correcting codes and to reconstruction of signals from few linear measurements. An error correcting code encodes an n-letter word x into an m-letter word y in such a way that x can be decoded correctly when any r letters of y are corrupted. We prove that most linear orthogonal transformations Q from R^n into R^m form efficient and robust robust error correcting codes over reals. The decoder (which corrects the corrupted components of y) is the metric projection onto the range of Q in the L_1 norm. An equivalent problem arises in signal processing: how to reconstruct a signal that belongs to a small class from few linear measurements? We prove that for most sets of Gaussian measurements, all signals of small support can be exactly reconstructed by the L_1 norm minimization. This is a substantial improvement of recent results of Donoho and of Candes and Tao. An equivalent problem in combinatorial geometry is the existence of a polytope with fixed number of facets and maximal number of lower-dimensional facets. We prove that most sections of the cube form such polytopes.

Motivation & Objective

• To develop a geometric functional analysis framework for error-correcting codes over the reals.
• To establish conditions under which the $\ell_1$-norm minimization (Basis Pursuit) exactly recovers signals from corrupted or incomplete measurements.
• To prove that most random linear transforms yield robust and efficient error-correcting codes.
• To connect signal recovery with the geometry of random subspaces and Gaussian measures on polytopes.
• To extend results to finite alphabets via quantization, showing robustness and continuity of the coding scheme.

Proposed method

• Uses the metric projection onto the range of a random linear map $Q$ in the $\ell_1$-norm as the decoding mechanism.
• Applies tools from asymptotic convex geometry and Gaussian measure concentration to analyze the geometry of subspaces and projections.
• Employs Anderson's Lemma and log-concavity of Gaussian measures to bound the measure of projections of the cube onto random subspaces.
• Analyzes the Grassmannian of $n$-dimensional subspaces in $\mathbb{R}^m$ with respect to the normalized Haar measure to establish probabilistic guarantees.
• Uses Minkowski addition and decomposition of index sets to control the size of projections of the $\ell_\infty$-ball.
• Establishes robustness of the $\ell_1$-based decoder via stability bounds: $\|u - y\|_1 \leq 4\|h\|_1$ for small perturbations $h$.

Experimental results

Research questions

• RQ1 Can $\ell_1$-minimization exactly reconstruct a sparse signal from a small number of linear measurements?
• RQ2 What is the minimal number of measurements $m$ required for exact recovery of $n$-dimensional signals with $r$-sparse corruption or support?
• RQ3 Do most random linear transforms $Q: \mathbb{R}^n \to \mathbb{R}^m$ yield robust error-correcting codes over the reals?
• RQ4 How does the geometry of random subspaces relate to the existence of polytopes with many low-dimensional faces?
• RQ5 Can the $\ell_1$-based reconstruction method be extended to finite alphabets with quantization while preserving error correction?

Key findings

• For most $n$-dimensional subspaces $Y \subset \mathbb{R}^m$, the $\ell_1$-projection decoder recovers any $y \in Y$ from a corrupted version $y'$ with up to $r$ corrupted coordinates, provided $m \gtrsim r \log(n/r)$.
• The probability of failure in reconstruction decays exponentially in $R = m/r$, with $P \leq 2e^{-cR}$ for $R \leq c \log(m/r)$, indicating high reliability.
• The $\ell_1$-based decoder is robust: $\|u - y\|_1 \leq 4\|h\|_1$ for any perturbation $h$, ensuring stability under small errors.
• Most $n$-dimensional subspaces of $\mathbb{R}^m$ yield error-correcting codes with $m \gtrsim r \log(n/r)$, matching the information-theoretic lower bound up to a logarithmic factor.
• The method extends to finite alphabets via quantization: the code recovers $x \in \{1,\dots,p\}^n$ from $\hat{y}$ with up to $r$ corrupted coordinates, provided $m \leq 2n$ and quantization step is $1/(10m)$.
• The Gaussian measure of the projection of the $\ell_\infty^n$-ball onto a random $n$-dimensional subspace $H$ satisfies $\gamma_H\left(\frac{c}{\sqrt{k}}\sqrt{\log(n/k)} P_H B_\infty^n\right) \leq e^{-cn/k}$, implying concentration and small measure for large $k$.