[Paper Review] Sequential decoding for lossless streaming source coding with side information

Hari Palaiyanur, Anant Sahai|ArXiv.org|Mar 23, 2007

Wireless Communication Security Techniques25 references5 citations

TL;DR

This paper proposes a sequential decoding scheme using random time-varying tree codes and a Stack Algorithm with variable bias for lossless streaming source coding with side information at the decoder. It achieves an exponentially decaying error probability with delay, matching Gallager’s random coding exponent, and ensures finite moments of computation under appropriate bias settings, extending to joint source-channel coding with similar performance guarantees.

ABSTRACT

The problem of lossless fixed-rate streaming coding of discrete memoryless sources with side information at the decoder is studied. A random time-varying tree-code is used to sequentially bin strings and a Stack Algorithm with a variable bias uses the side information to give a delay-universal coding system for lossless source coding with side information. The scheme is shown to give exponentially decaying probability of error with delay, with exponent equal to Gallager's random coding exponent for sources with side information. The mean of the random variable of computation for the stack decoder is bounded, and conditions on the bias are given to guarantee a finite $ρ^{th}$ moment for $0 \leq ρ\leq 1$. Further, the problem is also studied in the case where there is a discrete memoryless channel between encoder and decoder. The same scheme is slightly modified to give a joint-source channel encoder and Stack Algorithm-based sequential decoder using side information. Again, by a suitable choice of bias, the probability of error decays exponentially with delay and the random variable of computation has a finite mean. Simulation results for several examples are given.

Motivation & Objective

• To design a delay-universal, fixed-rate lossless source coding system for discrete memoryless sources when side information is available only at the decoder.
• To achieve an error exponent matching Gallager’s random coding exponent for sources with side information using sequential decoding.
• To bound the mean and higher moments of the computation random variable in the Stack Algorithm under variable bias.
• To extend the scheme to joint source-channel coding with side information, maintaining exponential error decay and finite computation moments.
• To provide conditions on the bias parameter that ensure both finite computation moments and positive error exponent.

Proposed method

• A random time-varying tree-code is used to sequentially bin source strings, enabling fixed-rate compression without encoder-side information.
• The Stack Algorithm with a variable bias parameter is employed at the decoder to exploit side information and decode source sequences sequentially.
• The bias is tuned to balance error exponent and moments of computation, with theoretical bounds derived using Gallager’s error exponent framework.
• Key equations involve the computation of error exponent terms $ E_{si}( heta) $, $ F_{si}( heta) $, and $ G_{si}( heta) $, derived from conditional entropy and likelihood ratios.
• For joint source-channel coding, the scheme is adapted by incorporating channel reliability via $ E_0( heta) $, $ F( heta) $, and $ G( heta) $, with bias adjusted to maintain performance.
• Jensen’s inequality and strict concavity of the log function are used to derive strict inequalities ensuring finite $ ho $-th moments for $ 0 \leq \rho \leq 1 $.

Experimental results

Research questions

• RQ1 Can a sequential decoding scheme achieve exponential error decay with delay in lossless source coding with decoder-only side information?
• RQ2 What conditions on the bias parameter ensure finite $ \rho $-th moments of the computation random variable for $ 0 \leq \rho \leq 1 $?
• RQ3 Can the scheme be extended to joint source-channel coding with side information while preserving exponential error decay and finite computation?
• RQ4 Is the error exponent of the proposed scheme equivalent to Gallager’s random coding exponent for sources with side information?
• RQ5 Does the choice of bias $ G^* = \frac{1+\gamma}{\gamma}[E_{si}(\gamma) - F_{si}(\gamma)] $ yield both positive error exponent and finite $ \gamma $-th moment of computation?

Key findings

• The proposed scheme achieves an error probability that decays exponentially with delay, with exponent equal to Gallager’s random coding exponent for sources with side information.
• The mean of the computation random variable in the Stack decoder is bounded under appropriate bias settings.
• For $ 0 \leq \rho \leq 1 $, the $ \rho $-th moment of computation is finite if the bias lies in the non-empty open interval $ \left( \frac{1+\gamma}{\gamma}G_{si}(\gamma), \frac{1+\gamma}{\gamma}(\gamma R - F_{si}(\gamma)) \right) $.
• In the joint source-channel coding case, a similar bias regime ensures exponential error decay and finite computation moments, provided $ \lambda E_0(\gamma) > E_{si}(\gamma) $.
• The optimal bias $ G^* = \frac{1+\gamma}{\gamma}[E_{si}(\gamma) - F_{si}(\gamma)] $ yields both a positive error exponent and finite $ \gamma $-th moment of computation, assuming $ U $ is not deterministic given $ v \in \mathcal{V} $ for at least one $ v $.
• The strict inequality $ G_{si}(\gamma) + F_{si}(\gamma) < E_{si}(\gamma) $ holds due to strict concavity of the log function and non-deterministic source, ensuring the bias interval is non-empty.