[Paper Review] Bounds on the decoding complexity of punctured codes on graphs

Henry D. Pfister, Igal Sason+1 more|ArXiv.org|Sep 14, 2004

Error Correcting Code Techniques11 references3 citations

TL;DR

This paper presents two sequences of non-systematic irregular repeat-accumulate (IRA) codes that achieve capacity on the binary erasure channel (BEC) with bounded decoding complexity per information bit, even as the rate approaches capacity. By puncturing all information bits to introduce state nodes in the Tanner graph, the method breaks the conventional trade-off between performance and complexity, enabling bounded complexity under message-passing iterative (MPI) decoding—unlike prior capacity-achieving codes whose complexity grows logarithmically with the gap to capacity.

ABSTRACT

We present two sequences of ensembles of non-systematic irregular repeat-accumulate codes which asymptotically (as their block length tends to infinity) achieve capacity on the binary erasure channel (BEC) with bounded complexity per information bit. This is in contrast to all previous constructions of capacity-achieving sequences of ensembles whose complexity grows at least like the log of the inverse of the gap (in rate) to capacity. The new bounded complexity result is achieved by puncturing bits, and allowing in this way a sufficient number of state nodes in the Tanner graph representing the codes. We also derive an information-theoretic lower bound on the decoding complexity of randomly punctured codes on graphs. The bound holds for every memoryless binary-input output-symmetric channel, and is refined for the BEC.

Motivation & Objective

• To address the fundamental trade-off between decoding complexity and performance in capacity-achieving codes on the BEC.
• To investigate whether state nodes in Tanner graphs can enable bounded decoding complexity under MPI decoding as the rate approaches capacity.
• To construct new ensembles of non-systematic IRA codes that achieve capacity with bounded complexity, overcoming limitations of systematic IRA codes.
• To derive an information-theoretic lower bound on decoding complexity for randomly punctured codes on graphs, valid for all memoryless binary-input output-symmetric (MBIOS) channels.
• To demonstrate that bounded complexity is only achievable when the puncturing rate of information bits approaches one, implying non-systematic design is essential.

Proposed method

• Puncturing all information bits in IRA codes to introduce state nodes in the Tanner graph, enabling bounded decoding complexity under MPI decoding.
• Designing non-systematic IRA code ensembles with irregular degree distributions (d.d.) that asymptotically achieve BEC capacity.
• Using a check-regular structure with check-node degree 5 to ensure constant decoding complexity per information bit of 5/(1−p) as the gap to capacity vanishes.
• Applying information-theoretic tools to derive a lower bound on decoding complexity for randomly punctured codes on graphs, valid for all MBIOS channels.
• Refining the bound specifically for the BEC, showing that bounded complexity requires the puncturing rate of information bits to approach one.
• Validating the analytical results through computer simulations comparing performance to check-regular LDPC codes and systematic IRA codes.

Experimental results

Research questions

• RQ1 Can capacity-achieving codes on the BEC be constructed with bounded decoding complexity per information bit under MPI decoding?
• RQ2 What is the role of state nodes in Tanner graphs in enabling bounded decoding complexity when approaching capacity?
• RQ3 Is there a fundamental lower bound on decoding complexity for randomly punctured codes on graphs, and how does it depend on channel type?
• RQ4 Why do systematic IRA codes fail to achieve bounded complexity as the rate approaches capacity, while non-systematic ones succeed?
• RQ5 Does the puncturing pattern of information bits fundamentally affect the possibility of achieving bounded complexity under MPI decoding?

Key findings

• The proposed non-systematic IRA code ensembles achieve capacity on the BEC with bounded decoding complexity per information bit, specifically 5/(1−p) for check-node degree 5, even as the gap to capacity vanishes.
• Puncturing all information bits introduces state nodes in the Tanner graph, which is essential for achieving bounded complexity—this mechanism breaks the prior complexity-rate trade-off.
• The information-theoretic lower bound on decoding complexity for randomly punctured codes on graphs shows that bounded complexity is only possible when the puncturing rate of information bits approaches one.
• For the BEC, the lower bound implies that a necessary condition for bounded complexity is that all information bits are punctured, which is consistent with the non-systematic design of the proposed codes.
• Simulation results confirm the analytical complexity bound and show that while performance is slightly worse than check-regular LDPC codes for small-to-moderate block lengths, the new codes eventually outperform all prior constructions as block length increases.
• The results resolve an open question from [9] by demonstrating that more complex graphical models (with state nodes) can significantly improve the performance-complexity trade-off under MPI decoding.