[Paper Review] Quantization of Binary-Input Discrete Memoryless Channels, with Applications to LDPC Decoding

Brian M. Kurkoski, Hideki Yagi|arXiv (Cornell University)|Jul 28, 2011

Error Correcting Code Techniques20 references14 citations

TL;DR

This paper proposes an optimal quantization method for binary-input discrete memoryless channels by maximizing mutual information using a quadratic-complexity dynamic programming algorithm. Applied to LDPC decoding, it systematically designs message-passing maps that achieve noise thresholds comparable to belief propagation using only four bits per message, outperforming heuristic quantized decoding approaches.

ABSTRACT

Abstract—The quantization of the output of a binary-input discrete memoryless channel to a smaller number of levels is considered. The optimal quantizer, in the sense of maximizing mutual information between the channel input and the quantizer output, may be found by an algorithm with complexity which is quadratic in the number of channel outputs. This is a concave optimization problem, and results from the field of concave optimization are invoked. The quantizer design algorithm is a realization of a dynamic program. Then, this algorithm is applied to the design of message-passing decoders for low-density parity-check codes, over arbitrary discrete memoryless channels. A general, systematic method to find message-passing decoding maps which maximize mutual information at each iteration is given. This may contrasted with existing quantized message-passing algorithms which are heuristically derived. The method finds message-passing decoding maps similar to those given by Richardson and Urbanke’s Algorithm E. Using four bits per message, noise thresholds similar to belief-propagation decoding are obtained. Index Terms—discrete memoryless channel, channel quantiza-tion, mutual information maximization, LDPC decoding I.

Motivation & Objective

• To develop an optimal quantizer for binary-input discrete memoryless channels that maximizes mutual information between input and quantized output.
• To address the lack of systematic design methods for message-passing decoding maps in quantized LDPC decoders.
• To provide a general, principled framework for deriving message-passing maps that maximize mutual information at each decoding iteration.
• To achieve performance close to belief propagation using minimal message precision, specifically four bits per message.

Proposed method

• The quantizer is designed using a dynamic programming algorithm that solves a concave optimization problem, ensuring global optimality in mutual information maximization.
• The algorithm operates in quadratic time relative to the number of channel output levels, leveraging results from concave optimization theory.
• The method formulates message-passing decoding maps as quantized messages that maximize mutual information at each iteration, derived from the optimal quantizer design.
• It generalizes and systematizes existing heuristic quantization rules, such as those in Richardson and Urbanke’s Algorithm E, by deriving them from information-theoretic principles.
• The approach is applied to arbitrary discrete memoryless channels, enabling systematic design of quantized decoders for LDPC codes.
• The method enables the design of message-passing decoders with minimal message precision—specifically four bits per message—while preserving high performance.

Experimental results

Research questions

• RQ1 What is the optimal way to quantize the output of a binary-input discrete memoryless channel to maximize mutual information between input and output?
• RQ2 How can message-passing decoding maps for LDPC codes be systematically derived to maximize mutual information at each iteration?
• RQ3 Can a principled, information-theoretic approach outperform heuristic quantization rules in LDPC decoding?
• RQ4 What message precision (in bits) is sufficient to achieve noise thresholds close to those of full-precision belief propagation?
• RQ5 How does the proposed quantization method compare to existing algorithms like Richardson and Urbanke’s Algorithm E in terms of performance and design rigor?

Key findings

• The proposed quantization algorithm achieves optimal mutual information maximization with quadratic complexity in the number of channel output levels.
• The method produces message-passing decoding maps that are systematically derived from information-theoretic principles, contrasting with heuristic approaches.
• Using only four bits per message, the method attains noise thresholds nearly identical to those of full-precision belief-propagation decoding.
• The dynamic programming formulation ensures global optimality in quantizer design, avoiding local minima common in heuristic methods.
• The approach generalizes to arbitrary discrete memoryless channels, enabling systematic design of quantized LDPC decoders.
• The results demonstrate that mutual information maximization leads to decoding maps that closely resemble those in Richardson and Urbanke’s Algorithm E, validating the method’s effectiveness.