[Paper Review] Single-Letter Characterization of Epsilon-Capacity for Mixed Memoryless Channels

Hideki Yagi, Ryo Nomura|arXiv (Cornell University)|Jul 1, 2014

Advanced Memory and Neural Computing3 citations

TL;DR

This paper provides the first single-letter characterization of ε-capacity for mixed memoryless channels composed of at most countably many stationary memoryless channels with a finite input alphabet and general output alphabet, under both unconstrained and cost-constrained settings. The characterization reduces to Ahlswede's channel capacity formula when ε = 0, and is proven using the information spectrum method combined with a particularized meta-converse bound from Polyanskiy, Poor, and Verdú.

ABSTRACT

For the class of mixed channels decomposed into stationary memoryless channels, single-letter characterizations of the $\varepsilon$-capacity have not been known except for restricted classes of channels such as the regular decomposable channel introduced by Winkelbauer. This paper gives single-letter characterizations of $\varepsilon$-capacity for mixed channels decomposed into at most countably many memoryless channels with a finite input alphabet and a general output alphabet with/without cost constraints. It is shown that a given characterization reduces to the one for the channel capacity given by Ahlswede when $\varepsilon$ is zero. In the proof of the coding theorem, the meta converse bound, originally given by Polyanskiy, Poor and Verdú, is particularized for the mixed channel decomposed into general component channels.

Motivation & Objective

• To close a longstanding gap in information theory by providing a single-letter characterization of ε-capacity for mixed memoryless channels.
• To extend known results beyond the restricted class of regular decomposable channels studied by Winkelbauer.
• To establish a rigorous, computable expression for ε-capacity that avoids limit operations over blocklength n.
• To incorporate cost constraints into the ε-capacity characterization for mixed channels.
• To validate the characterization through a coding theorem using the meta-converse bound and information spectrum methods.

Proposed method

• Derives a single-letter expression for ε-capacity using the information spectrum method and a particularized version of the meta-converse bound for mixed channels.
• Applies the duality between α and β in hypothesis testing to bound the error probability via the αβ function.
• Uses the maximum likelihood decoder's error probability as a lower bound to derive a converse bound on achievable rates.
• Introduces a mixture of component channels with weights {wℓ} to model the overall mixed channel behavior.
• Employs a concentration argument based on Chebyshev's inequality to control deviations in the log-likelihood ratio of the channel output.
• Establishes the converse by showing that any code with rate exceeding the proposed expression must violate the ε-error constraint.

Experimental results

Research questions

• RQ1 Can a single-letter expression for ε-capacity be derived for mixed memoryless channels with a finite input alphabet and general output alphabet, beyond restricted classes like regular decomposable channels?
• RQ2 Does the proposed single-letter characterization reduce to Ahlswede's channel capacity formula when ε = 0?
• RQ3 How can the meta-converse bound be adapted and applied to mixed channels to establish tight converse bounds?
• RQ4 Can the characterization be extended to include input cost constraints in the mixed channel setting?
• RQ5 Is the resulting ε-capacity expression computable without limit operations over blocklength n?

Key findings

• The paper establishes a single-letter expression for ε-capacity of mixed memoryless channels with at most countably many component channels, valid for both unconstrained and cost-constrained input scenarios.
• The proposed characterization reduces to Ahlswede’s single-letter channel capacity formula when ε = 0, confirming consistency with classical results.
• The converse bound is derived using a particularized version of the meta-converse bound, which is shown to be tight for mixed channels.
• The coding theorem is proven via the information spectrum method, with error probability bounds derived using hypothesis testing duality and concentration inequalities.
• The characterization is shown to be computable with complexity independent of blocklength n, unlike previous limit-based formulas.
• The result extends the applicability of single-letter formulas to non-ergodic, non-stationary settings such as block-fading channels modeled as mixed channels.