[Paper Review] Fixed Error Asymptotics For Erasure and List Decoding

Vincent Y. F. Tan, Pierre Moulin|arXiv (Cornell University)|Feb 20, 2014

Wireless Communication Security Techniques36 references5 citations

TL;DR

This paper derives second-order asymptotics for erasure and list decoding in discrete memoryless channels, showing that the second-order capacity for erasure decoding is $\sqrt{V}\Phi^{-1}(\epsilon_{\mathrm{t}})$, where $V$ is channel dispersion and $\epsilon_{\mathrm{t}}$ is the fixed total error probability. For list decoding with list size $\exp(\sqrt{n}l)$, the second-order capacity is $l + \sqrt{V}\Phi^{-1}(\epsilon)$, with bounds on third-order terms for polynomial-sized lists.

ABSTRACT

We derive the optimum second-order coding rates, known as second-order capacities, for erasure and list decoding. For erasure decoding for discrete memoryless channels, we show that second-order capacity is $\sqrt{V}Φ^{-1}(ε_t)$ where $V$ is the channel dispersion and $ε_t$ is the total error probability, i.e., the sum of the erasure and undetected errors. We show numerically that the expected rate at finite blocklength for erasures decoding can exceed the finite blocklength channel coding rate. We also show that the analogous result also holds for lossless source coding with decoder side information, i.e., Slepian-Wolf coding. For list decoding, we consider list codes of deterministic size that scales as $\exp(\sqrt{n}l)$ and show that the second-order capacity is $l+\sqrt{V}Φ^{-1}(ε)$ where $ε$ is the permissible error probability. We also consider lists of polynomial size $n^α$ and derive bounds on the third-order coding rate in terms of the order of the polynomial $α$. These bounds are tight for symmetric and singular channels. The direct parts of the coding theorems leverage on the simple threshold decoder and converses are proved using variants of the hypothesis testing converse.

Motivation & Objective

• To establish second-order asymptotic limits for erasure and list decoding under fixed, non-vanishing error probabilities.
• To determine the second-order capacity for erasure decoding, showing it depends only on total error probability $\epsilon_{\mathrm{t}}$, not undetected error probability.
• To extend the analysis to list decoding with non-constant list sizes, including exponential and polynomial growth rates.
• To derive finite-blocklength performance bounds and show that expected rates with erasures can exceed standard channel coding rates.
• To generalize results to the Slepian-Wolf source coding problem with decoder side information.

Proposed method

• Derives second-order capacity for erasure decoding using a threshold decoder and hypothesis testing converse techniques.
• Applies the Berry-Esseen theorem to approximate the distribution of empirical conditional entropy and bound error probabilities.
• Uses type class enumeration and binning arguments to bound undetected error probabilities in the Slepian-Wolf setting.
• Introduces a threshold-based erasure rule based on empirical conditional entropy $\hat{H}(\mathbf{x}|\mathbf{y})$ to control total error probability.
• Analyzes list decoding with list sizes $\exp(\sqrt{n}l)$ and $n^\alpha$, deriving second- and third-order asymptotics via information-spectrum methods.
• Employs Markov inequality and concentration bounds to control error probabilities and establish achievability.

Experimental results

Research questions

• RQ1 What is the second-order capacity for erasure decoding when the total error probability $\epsilon_{\mathrm{t}}$ is fixed?
• RQ2 Can the expected rate in finite-blocklength erasure decoding exceed the standard channel coding rate?
• RQ3 What is the second-order capacity for list decoding with list size $\exp(\sqrt{n}l)$?
• RQ4 How do third-order coding rates behave for polynomial-sized lists $n^\alpha$?
• RQ5 Do the second- and third-order asymptotics for erasure and list decoding extend to the Slepian-Wolf source coding problem with side information?

Key findings

• The second-order capacity for erasure decoding is $\sqrt{V}\Phi^{-1}(\epsilon_{\mathrm{t}})$, independent of the undetected error probability, where $V$ is the channel dispersion and $\epsilon_{\mathrm{t}}$ is the total error probability.
• Finite-blocklength erasure decoding can achieve a higher expected rate than standard channel coding without erasures, due to reduced error probability and improved reliability.
• For list decoding with list size $\exp(\sqrt{n}l)$, the second-order capacity is $l + \sqrt{V}\Phi^{-1}(\epsilon)$, where $\epsilon$ is the probability that the true message is not in the list.
• For polynomial-sized lists $n^\alpha$, the third-order coding rate is bounded, and these bounds are tight for symmetric and singular channels such as the binary erasure channel.
• The results extend to Slepian-Wolf source coding with side information, where the second-order capacity is $\sqrt{V(X|Y)}\Phi^{-1}(1 - \epsilon_{\mathrm{t}})$ for fixed total error probability $\epsilon_{\mathrm{t}}$.
• The converse proofs rely on variants of the hypothesis testing converse, and achievability is shown via threshold decoding and type-based analysis with binning.