[Paper Review] Universal Sparse Superposition Codes with Spatial Coupling and GAMP Decoding

Jean Barbier, Mohamad Dia+1 more|arXiv (Cornell University)|Jul 13, 2017

Error Correcting Code Techniques4 citations

TL;DR

This paper introduces spatially coupled sparse superposition codes that universally achieve capacity over a broad class of memoryless channels under generalized approximate message-passing (GAMP) decoding. By leveraging state evolution analysis and the potential method, it proves threshold saturation—where the algorithmic threshold of the coupled system reaches the Bayes-optimal (potential) threshold—demonstrating universal capacity-achieving performance as code parameters grow large.

ABSTRACT

Sparse superposition codes, or sparse regression codes, constitute a new class of codes which was first introduced for communication over the additive white Gaussian noise (AWGN) channel. It has been shown that such codes are capacity-achieving over the AWGN channel under optimal maximum-likelihood decoding as well as under various efficient iterative decoding schemes equipped with power allocation or spatially coupled constructions. Here, we generalize the analysis of these codes to a much broader setting that includes all memoryless channels. We show, for a large class of memoryless channels, that spatial coupling allows an efficient decoder, based on the generalized approximate message-passing (GAMP) algorithm, to reach the potential (or Bayes optimal) threshold of the underlying (or uncoupled) code ensemble. Moreover, we argue that spatially coupled sparse superposition codes universally achieve capacity under GAMP decoding by showing, through analytical computations, that the error floor vanishes and the potential threshold tends to capacity as one of the code parameter goes to infinity. Furthermore, we provide a closed form formula for the algorithmic threshold of the underlying code ensemble in terms of a Fisher information. Relating an algorithmic threshold to a Fisher information has theoretical as well as practical importance. Our proof relies on the state evolution analysis and uses the potential method developed in the theory of low-density parity-check (LDPC) codes and compressed sensing.

Motivation & Objective

• To extend the analysis of sparse superposition codes beyond the AWGN channel to a broad class of memoryless channels.
• To demonstrate that spatial coupling enables GAMP decoding to achieve the Bayes-optimal threshold (potential threshold) of the underlying code ensemble.
• To prove that as the code parameter grows, the error floor vanishes and the potential threshold approaches capacity, establishing universal capacity-achieving performance.
• To derive a closed-form expression for the algorithmic threshold in terms of Fisher information, linking algorithmic performance to a fundamental information-theoretic quantity.
• To unify the analysis of sparse superposition codes across different channels using state evolution and the potential method, extending techniques from compressed sensing and LDPC codes.

Proposed method

• Proposes a spatially coupled construction of sparse superposition codes using a coupling matrix $ J_{r,c} $, where code symbols are connected across spatial positions to enhance decoding performance.
• Applies the generalized approximate message-passing (GAMP) algorithm for iterative decoding, with state evolution equations tracking the mean-squared error of estimates across iterations.
• Uses the potential method—originally developed for LDPC codes and compressed sensing—to analyze the coupled system, enabling rigorous threshold saturation proofs.
• Derives the algorithmic threshold as the solution to a variational problem involving the potential function, which is expressed in terms of Fisher information of the channel output.
• Performs asymptotic analysis in the large alphabet size limit, showing that the potential threshold converges to Shannon capacity.
• Establishes a closed-form expression for the algorithmic threshold via the Fisher information of the channel, linking it to the optimal performance of the underlying code ensemble.

Experimental results

Research questions

• RQ1 Can spatially coupled sparse superposition codes universally achieve capacity across a broad class of memoryless channels under GAMP decoding?
• RQ2 Does threshold saturation occur in spatially coupled sparse superposition codes, such that the algorithmic threshold of GAMP decoding reaches the Bayes-optimal (potential) threshold?
• RQ3 How does the algorithmic threshold of the coupled system relate to the Fisher information of the channel, and can this relationship be expressed in closed form?
• RQ4 What is the asymptotic behavior of the potential threshold as the code parameter (e.g., alphabet size) increases, and does it converge to Shannon capacity?
• RQ5 Can the potential method, previously used in compressed sensing and LDPC codes, be successfully extended to dense coding matrices and general memoryless channels in the context of sparse superposition codes?

Key findings

• Spatially coupled sparse superposition codes achieve the potential (Bayes-optimal) threshold of the underlying code ensemble under GAMP decoding, proving threshold saturation.
• The algorithmic threshold of the coupled system is expressed in closed form as a function of the Fisher information of the channel, providing a direct link between information geometry and decoding performance.
• As the code parameter (e.g., input alphabet size) increases, the potential threshold converges to Shannon capacity, demonstrating universal capacity-achieving performance across memoryless channels.
• The error floor vanishes in the large code limit, confirming the robustness and reliability of the decoding process under spatial coupling.
• The state evolution analysis is exact for the GAMP algorithm in this setting, validating the use of the potential method for performance prediction.
• The theoretical framework unifies the analysis of sparse superposition codes across different channels, including AWGN, binary symmetric, binary erasure, and Z-channels, with consistent performance guarantees.