[Paper Review] Statistical Mechanical Analysis of Low-Density Parity-Check Codes on General Markov Channel

Ryuhei Mori, Toshiyuki Tanaka|arXiv (Cornell University)|Oct 10, 2011

Error Correcting Code Techniques21 references3 citations

TL;DR

This paper applies statistical mechanics, specifically the replica method, to analyze low-density parity-check (LDPC) codes on general Markov and asymmetric memoryless channels. It derives saddle-point equations that exactly match the density evolution equations of belief propagation decoding, establishing a rigorous connection between statistical physics and iterative decoding thresholds, with explicit results for the binary symmetric and binary erasure channels showing agreement with known bounds.

ABSTRACT

Low-density parity-check (LDPC) codes on symmetric memoryless channels have been analyzed using statistical physics by several authors. In this paper, statistical mechanical analysis of LDPC codes is performed for asymmetric memoryless channels and general Markov channels. It is shown that the saddle point equations of the replica symmetric solution for a Markov channel is equivalent to the density evolution of the belief propagation on the factor graph representing LDPC codes on the Markov channel. The derivation uses the method of types for Markov chain.

Motivation & Objective

• To extend statistical mechanical analysis of LDPC codes from symmetric memoryless channels to general asymmetric and Markovian channels.
• To establish a rigorous connection between the replica method and belief propagation decoding via density evolution.
• To derive analytical expressions for the expected conditional entropy and decoding thresholds using the method of types for Markov chains.
• To validate the replica method's predictions against known bounds and numerical results, particularly for the binary erasure channel (BEC) and binary symmetric channel (BSC).

Proposed method

• The replica method is applied to compute the quenched average of the log-partition function, representing the expected conditional entropy of codewords given channel outputs.
• The method of types for Markov chains is used to handle the memory in the channel, generalizing previous results on i.i.d. inputs.
• Saddle-point equations are derived for the replica symmetric solution, which are shown to be mathematically equivalent to the density evolution equations of belief propagation on the Tanner graph.
• The analysis uses a cavity method approach to compute message-passing dynamics, with order parameters defined over variable and check node messages.
• The replica symmetric free energy is extremized over message distributions, leading to a set of self-consistent equations for the error probabilities.
• The resulting equations are solved numerically for regular LDPC ensembles, yielding the expected conditional entropy and decoding thresholds.

Experimental results

Research questions

• RQ1 Can the replica method be extended to analyze LDPC codes on general Markov channels with memory and asymmetry?
• RQ2 Is the replica symmetric solution for LDPC codes on Markov channels equivalent to the density evolution equations of belief propagation?
• RQ3 What are the thresholds for belief propagation and maximum a posteriori (MAP) decoding on asymmetric and Markovian channels?
• RQ4 How do the BP and MAP thresholds compare across different regular LDPC code ensembles on the binary erasure channel (BEC)?
• RQ5 Does the replica method yield consistent results with known analytical bounds, such as the Shannon limit, for rate-half codes?

Key findings

• The saddle-point equations derived from the replica method exactly match the density evolution equations of belief propagation on the Tanner graph for LDPC codes over general Markov channels.
• For the binary erasure channel (BEC), the replica method predicts a BP threshold of approximately 0.56891 and a MAP threshold of 0.63865 for the (3,6)-regular LDPC ensemble.
• The (2,4)-regular LDPC ensemble exhibits a higher BP threshold than the (3,6)-ensemble, contrary to the behavior on memoryless channels.
• The expected conditional entropy becomes positive only above the MAP threshold, indicating successful decoding, while negative values in some regions suggest the need to select the trivial saddle point for non-negative entropy.
• The results confirm that the upper bound on the MAP threshold from previous work is tight, and the replica method yields results consistent with known Shannon limits for rate-half codes.