[Paper Review] Single-bit Quantization Capacity of Binary-input Continuous-output Channels

Thuan Nguyen, Thinh Nguyen|arXiv (Cornell University)|Jan 7, 2020

Error Correcting Code Techniques20 references4 citations

TL;DR

This paper proposes a linear-time algorithm to jointly optimize the input distribution and 1-bit quantizer threshold for binary-input continuous-output channels, maximizing mutual information I(X;Z). By exploiting analytical properties of the optimal quantizer and input distribution, the method reduces the search space, enabling efficient global optimization with proven convergence and a key result of I(X;Z)* = 0.7249 under specified Gaussian noise conditions.

ABSTRACT

We consider a channel with discrete binary input X that is corrupted by a given continuous noise to produce a continuous-valued output Y. A quantizer is then used to quantize the continuous-valued output Y to the final binary output Z. The goal is to design an optimal quantizer Q* and also find the optimal input distribution p*(X) that maximizes the mutual information I(X; Z) between the binary input and the binary quantized output. A linear time complexity searching procedure is proposed. Based on the properties of the optimal quantizer and the optimal input distribution, we reduced the searching range that results in a faster implementation algorithm. Both theoretical and numerical results are provided to illustrate our method.

Motivation & Objective

• To solve the joint optimization of input distribution and 1-bit quantizer threshold in binary-input continuous-output channels to maximize mutual information I(X;Z).
• To overcome the non-convex/non-concave nature of mutual information in quantizer parameters, which prevents standard convex optimization techniques.
• To reduce the computational complexity of global optimization by deriving analytical bounds on the optimal solution space.
• To provide a provably convergent, linear-time complexity algorithm for practical implementation in communication systems.

Proposed method

• Proposes a bisection-based search over the likelihood ratio r = φ₀(y)/φ₁(y) to find optimal quantization thresholds.
• Derives analytical bounds on the optimal mutual information by proving that A₁₁* > 1/e and A₂₂* > 1/e, where Aᵢⱼ are entries of the optimal channel matrix.
• Uses monotonicity properties of f(r) (probability of output 0 given input 0) and g(r) (probability of output 1 given input 1) to narrow the search interval.
• Applies exhaustive search within the reduced interval [0.8, 9.1] with resolution ε = 0.01 to locate the global maximum of I(X;Z).
• Leverages the fact that f(r) is decreasing and g(r) is increasing in r, enabling efficient bisection to find bounds where f(r) = 1/e and g(r) = 1/e.
• Employs theoretical results from Theorems 3, 5, and 6 to constrain the feasible region for r, significantly reducing computational load.

Experimental results

Research questions

• RQ1 What is the global optimal input distribution and 1-bit quantizer threshold that maximizes mutual information I(X;Z) in a binary-input continuous-output channel?
• RQ2 How can the search space for the optimal quantizer threshold be analytically reduced to improve computational efficiency?
• RQ3 What are the necessary conditions on the channel matrix entries A₁₁ and A₂₂ for the optimal solution to exist?
• RQ4 Can a linear-time complexity algorithm be designed to achieve global optimality despite the non-convexity of the mutual information in quantizer parameters?

Key findings

• The optimal mutual information I(X;Z)* is 0.7249 for a channel with φ₀ ~ N(−1, 6) and φ₁ ~ N(1, 5), achieved at r* = 1.36.
• The optimal quantizer threshold corresponds to a likelihood ratio r* = 1.36, which lies within the reduced search interval [0.8, 9.1] derived from the condition A₁₁* > 1/e and A₂₂* > 1/e.
• The function f(r) (P(Z=0|X=0)) is strictly decreasing and g(r) (P(Z=1|X=1)) is strictly increasing in r, enabling efficient bisection search.
• The optimal input distribution satisfies A₁₁ ≥ p₀ and A₂₂ ≥ p₁, derived from the condition (A₁₁ − p₀)(A₂₂ − p₁) ≥ 0 and A₁₁ + A₂₂ ≥ 1.
• The search space reduction via theoretical bounds results in a significant speedup, enabling a linear-time complexity algorithm despite the non-convex nature of the problem.
• The method guarantees global optimality under the given constraints, unlike heuristic or near-optimal approaches that may fail or converge to suboptimal solutions.