[Paper Review] Performance versus Complexity Per Iteration for Low-Density Parity-Check Codes: An Information-Theoretic Approach

Igal Sason, Gil Wiechman|ArXiv.org|Dec 18, 2005

Error Correcting Code Techniques22 references3 citations

TL;DR

This paper presents an information-theoretic framework to analyze the tradeoff between decoding complexity per iteration and performance (gap to capacity) for low-density parity-check (LDPC) codes. By generalizing bounds to parallel channels and improving prior results, it quantifies the sub-optimality of message-passing iterative decoding relative to maximum-likelihood decoding, showing that over 1/3 of the gap to capacity in punctured codes stems from code structure, not decoding sub-optimality.

ABSTRACT

The paper is focused on the tradeoff between performance and decoding complexity per iteration for LDPC codes in terms of their gap (in rate) to capacity. The study of this tradeoff is done via information-theoretic bounds which also enable to get an indication on the sub-optimality of message-passing iterative decoding algorithms (as compared to optimal ML decoding). The bounds are generalized for parallel channels, and are applied to ensembles of punctured LDPC codes where both intentional and random puncturing are addressed. This work suggests an improvement in the tightness of some information-theoretic bounds which were previously derived by Burshtein et al. and by Sason and Urbanke.

Motivation & Objective

• To quantify the tradeoff between decoding complexity per iteration and performance (gap to capacity) for LDPC codes.
• To analyze the inherent performance loss due to sub-optimal message-passing iterative decoding compared to maximum-likelihood decoding.
• To improve existing information-theoretic bounds on parity-check density and decoding complexity for LDPC codes.
• To extend these bounds to ensembles of punctured LDPC codes under both intentional and random puncturing.
• To assess the contribution of code structure versus iterative decoding sub-optimality to the overall gap to capacity.

Proposed method

• Derives improved information-theoretic lower bounds on parity-check density for LDPC codes under memoryless binary-input output-symmetric (MBIOS) channels.
• Generalizes bounds to parallel channels to model punctured LDPC codes, enabling analysis of both intentional and random puncturing.
• Applies the bounds to ensembles of punctured LDPC codes, using degree distributions and puncturing patterns from prior work.
• Uses density evolution to compute exact thresholds for iterative decoding and compares them with the derived ML decoding lower bounds.
• Leverages the GEXIT curve matching condition to support conjectures on complexity scaling with gap to capacity.
• Improves upon prior bounds by Burshtein et al. and Sason and Urbanke, particularly for the binary erasure channel (BEC).

Experimental results

Research questions

• RQ1 What is the minimum parity-check density required for LDPC codes to achieve a given fraction of channel capacity, and how does this relate to decoding complexity per iteration?
• RQ2 How much of the performance gap to capacity in LDPC codes is due to code structure versus the sub-optimality of message-passing iterative decoding?
• RQ3 How do information-theoretic bounds on ML decoding performance compare to actual thresholds under iterative decoding for punctured LDPC codes?
• RQ4 Can the bounds be generalized to parallel channels to model punctured codes, and what insights do they provide on complexity-performance tradeoffs?
• RQ5 What is the scaling behavior of decoding complexity per information bit as the gap to capacity vanishes, and how does puncturing affect this?

Key findings

• For ensembles of intentionally punctured LDPC codes over the binary-input AWGN channel, more than 1/3 of the gap to capacity is attributed to code structure, even under optimal ML decoding.
• The proposed information-theoretic bounds provide tighter lower bounds on the required parity-check density compared to prior work by Burshtein et al. and Sason and Urbanke.
• The bounds indicate that the sub-optimality of message-passing iterative decoding accounts for a significant but not dominant part of the performance gap to capacity.
• For the binary erasure channel (BEC), the improved bound remains logarithmic in 1/ε, consistent with bounded complexity under both encoding and decoding.
• The analysis confirms that puncturing is essential for achieving capacity with bounded complexity per information bit, especially in ensembles of LDPC codes.
• The results support the conjecture that decoding complexity scales like 1/ε ln(1/ε) for general MBIOS channels, with the BEC being an exception due to message reliability.