[Paper Review] Rateless Codes for Finite Message Set

Navot Blits|arXiv (Cornell University)|Jan 27, 2012

Wireless Communication Security Techniques11 references3 citations

TL;DR

This paper proposes a non-asymptotic rateless coding framework for finite message sets over discrete memoryless channels (DMCs) with feedback, using sequential decoding to achieve variable-rate transmission with a fixed error probability. It establishes achievable rates that converge to channel capacity as the message set grows, and introduces a universal decoder using mixture probability assignment for unknown channels, enabling complete universality without prior knowledge of source, channel, or side information.

ABSTRACT

In this study we consider rateless coding over discrete memoryless channels (DMC) with feedback. Unlike traditional fixed-rate codes, in rateless codes each codeword is infinitely long, and the decoding time depends on the confidence level of the decoder. Using rateless codes along with sequential decoding, and allowing a fixed probability of error at the decoder, we obtain results for several communication scenarios. The results shown here are non-asymptotic, in the sense that the size of the message set is finite. First we consider the transmission of equiprobable messages using rateless codes over a DMC, where the decoder knows the channel law. We obtain an achievable rate for a fixed error probability and a finite message set. We show that as the message set size grows, the achievable rate approaches the optimum rate for this setting. We then consider the universal case, in which the channel law is unknown to the decoder. We introduce a novel decoder that uses a mixture probability assignment instead of the unknown channel law, and obtain an achievable rate for this case. Finally, we extend the scope for more advanced settings. We use different flavors of the rateless coding scheme for joint source-channel coding, coding with side-information and a combination of the two with universal coding, which yields a communication scheme that does not require any information on the source, the channel, or the amount the side information at the receiver.

Motivation & Objective

• To develop rateless coding schemes for finite message sets in point-to-point communication over DMCs with feedback, enabling non-asymptotic analysis.
• To address the challenge of achieving reliable communication when the channel law is unknown to the decoder, by introducing a novel universal decoding metric.
• To extend rateless coding to joint source-channel coding and source coding with side information, achieving optimal rates without prior knowledge of source statistics or side information.
• To quantify the convergence rate of achievable rates to capacity, particularly under uncertainty about channel and source statistics.
• To demonstrate that complete universality—no knowledge of source, channel, or side information—is feasible with rateless codes, maintaining asymptotic optimality.

Proposed method

• Uses sequential decoding with a stopping time based on Wald's sequential probability ratio test (SPRT) to dynamically terminate transmission when confidence in the message exceeds a threshold.
• Employs a known channel law in the likelihood ratio computation for the informed decoder case, enabling rate analysis via stopping time theory.
• Introduces a universal decoder that replaces the unknown channel law with a mixture probability assignment, bounding the difference between universal and informed metrics.
• Applies the derived bounds to upper bound mean transmission time in the universal case, leveraging results from the informed decoder setting.
• Adapts the sequential decoder for joint source-channel coding by incorporating source statistics into the likelihood computation, achieving Slepian-Wolf and Wyner-Ziv rates.
• Combines universal channel coding, joint source-channel coding, and side information techniques into a single universal scheme, requiring no prior knowledge of source, channel, or side information.

Experimental results

Research questions

• RQ1 What is the achievable rate of rateless codes for a finite message set over a DMC with feedback and a known channel law?
• RQ2 How does the performance of rateless coding degrade when the channel law is unknown, and can a universal decoder mitigate this loss?
• RQ3 Can rateless coding achieve the optimal Slepian-Wolf rate for source coding with side information at the decoder?
• RQ4 What is the rate of convergence of the achievable rate to channel capacity as the message set size increases?
• RQ5 Is it possible to design a completely universal rateless coding scheme that achieves the optimal source-channel coding rate without any knowledge of the source, channel, or side information?

Key findings

• For a known channel, the achievable rate approaches the channel capacity as the message set size $M$ grows, with convergence dominated by an $O(1/M)$ term.
• In the universal case with unknown channel, the achievable rate is $R = C ig(1 - O(rac{ ext{log } K}{K})ig) + O(rac{1}{K})$, where $K$ is the message set size, showing asymptotic optimality.
• The rate of convergence for the universal case is slowed by a $O(rac{ ext{log } K}{K})$ term due to lack of channel knowledge, compared to $O(rac{1}{K})$ for the informed case.
• For joint source-channel coding, the rateless scheme achieves the optimal separation-based rate, matching the capacity of the combined system.
• With side information at the decoder, the rate improves to $R = rac{C}{ ext{H}(W|V)} ig(1 - O(rac{ ext{log } K}{K})ig) + O(rac{1}{K})$, where $ ext{H}(W|V)$ is the conditional entropy.
• The proposed universal scheme achieves the optimal source-channel coding rate even without knowledge of the source distribution, channel law, or existence/amount of side information, with the same asymptotic convergence as the informed case.