[Paper Review] Dynamic Programming for Sequential Deterministic Quantization of Discrete Memoryless Channels

Xuan He, Kui Cai+2 more|arXiv (Cornell University)|Jan 7, 2019

Error Correcting Code Techniques26 references4 citations

TL;DR

This paper presents a dynamic programming (DP) framework for designing optimal sequential deterministic quantizers (SDQs) in q-ary input discrete memoryless channels (DMCs) under a general cost function C. It achieves complexity reduction via the SMAWK algorithm and a simpler technique, with applications to α-mutual information maximization and practical PAM systems.

ABSTRACT

In this paper, under a general cost function $C$, we present a dynamic programming (DP) method to obtain an optimal sequential deterministic quantizer (SDQ) for $q$-ary input discrete memoryless channel (DMC). The DP method has complexity $O(q (N-M)^2 M)$, where $N$ and $M$ are the alphabet sizes of the DMC output and quantizer output, respectively. Then, starting from the quadrangle inequality, two techniques are applied to reduce the DP method's complexity. One technique makes use of the Shor-Moran-Aggarwal-Wilber-Klawe (SMAWK) algorithm and achieves complexity $O(q (N-M) M)$. The other technique is much easier to be implemented and achieves complexity $O(q (N^2 - M^2))$. We further derive a sufficient condition under which the optimal SDQ is optimal among all quantizers and the two techniques are applicable. This generalizes the results in the literature for binary-input DMC. Next, we show that the cost function of $α$-mutual information ($α$-MI)-maximizing quantizer belongs to the category of $C$. We further prove that under a weaker condition than the sufficient condition we derived, the aforementioned two techniques are applicable to the design of $α$-MI-maximizing quantizer. Finally, we illustrate the particular application of our design method to practical pulse-amplitude modulation systems.

Motivation & Objective

• To develop an efficient dynamic programming method for optimal sequential deterministic quantization (SDQ) in q-ary input discrete memoryless channels (DMCs).
• To reduce computational complexity of the DP method using the quadrangle inequality and two optimization techniques: SMAWK algorithm and a simpler heuristic.
• To derive a sufficient condition under which the optimal SDQ is globally optimal among all quantizers, generalizing prior binary-input results.
• To show that α-mutual information (α-MI) maximization falls within the cost function class C, enabling application of the proposed methods.
• To demonstrate practical applicability in pulse-amplitude modulation (PAM) systems, particularly in AWGN and NVM channels.

Proposed method

• Proposes a dynamic programming formulation to compute the optimal SDQ by parameterizing the quantizer via thresholds λ₀=0 < λ₁ < ... < λₘ=N.
• Uses a cost function C(Q) that generalizes mutual information and other divergence measures, enabling broad applicability.
• Applies the quadrangle inequality (QI) to the cost function to enable complexity reduction via the SMAWK algorithm.
• Introduces two complexity-reduction techniques: one using the SMAWK algorithm to achieve O(q(N−M)M) complexity, and another simpler method achieving O(q(N²−M²)) complexity.
• Derives a sufficient condition for global optimality of the SDQ by analyzing the structure of the cost function and its submodularity properties.
• Proves that α-mutual information (α-MI) maximization satisfies the required conditions for the complexity-reduction techniques to apply under a weaker condition than the sufficient condition.

Experimental results

Research questions

• RQ1 Can an optimal sequential deterministic quantizer (SDQ) be efficiently computed for q-ary input DMCs under a general cost function?
• RQ2 Under what conditions can the computational complexity of the DP-based SDQ design be reduced using the SMAWK algorithm?
• RQ3 Is the α-mutual information (α-MI) maximizing quantizer design compatible with the proposed DP and complexity-reduction framework?
• RQ4 What is the sufficient condition under which the optimal SDQ is globally optimal among all possible quantizers?
• RQ5 How can the proposed method be applied to practical communication systems such as PAM-based modulation in AWGN and NVM channels?

Key findings

• The proposed DP method achieves optimal SDQ design with computational complexity O(q(N−M)²M) for q-ary input DMCs.
• Using the SMAWK algorithm, the complexity is reduced to O(q(N−M)M), significantly improving efficiency for large N and M.
• A simpler complexity-reduction technique achieves O(q(N²−M²)) complexity, offering a practical trade-off between performance and implementation ease.
• A sufficient condition is derived under which the optimal SDQ is globally optimal among all quantizers, generalizing prior binary-input results.
• The α-mutual information (α-MI) cost function is shown to be compatible with the framework, and the complexity-reduction techniques apply under a weaker condition than the sufficient condition.
• The method is demonstrated to be applicable to practical systems such as PAM-based modulation in AWGN and non-volatile memory (NVM) channels, validating its real-world relevance.