[Paper Review] Optimum Trade-offs Between the Error Exponent and the Excess-Rate Exponent of Variable-Rate Slepian-Wolf Coding

Nir Weinberger, Neri Merhav|arXiv (Cornell University)|Jan 5, 2014

Wireless Communication Security Techniques25 references7 citations

TL;DR

This paper establishes the optimal trade-off between error exponent and excess-rate exponent in variable-rate Slepian-Wolf coding by deriving tight upper (converse) and lower (achievability) bounds using type-based coding. The key contribution is a systematic analysis showing that type-class uniform rate assignment achieves a balanced performance between fixed-rate (low excess-rate) and average-rate (high reliability) coding, with explicit bounds on rate functions and excess-rate exponents via iterative algorithms.

ABSTRACT

We analyze the optimal trade-off between the error exponent and the excess-rate exponent for variable-rate Slepian-Wolf codes. In particular, we first derive upper (converse) bounds on the optimal error and excess-rate exponents, and then lower (achievable) bounds, via a simple class of variable-rate codes which assign the same rate to all source blocks of the same type class. Then, using the exponent bounds, we derive bounds on the optimal rate functions, namely, the minimal rate assigned to each type class, needed in order to achieve a given target error exponent. The resulting excess-rate exponent is then evaluated. Iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess-rate exponents. The resulting Slepian-Wolf codes bridge between the two extremes of fixed-rate coding, which has minimal error exponent and maximal excess-rate exponent, and average-rate coding, which has maximal error exponent and minimal excess-rate exponent.

Motivation & Objective

• To analyze the fundamental trade-off between error exponent and excess-rate exponent in variable-rate Slepian-Wolf coding.
• To derive upper (converse) and lower (achievable) bounds on the optimal error and excess-rate exponents for general variable-rate codes.
• To characterize the minimal rate function per type class required to achieve a target error exponent, while evaluating the resulting excess-rate exponent.
• To bridge the performance gap between fixed-rate coding (minimal excess-rate, low error exponent) and average-rate coding (maximal error exponent, minimal excess-rate) via a unified framework.

Proposed method

• Derives upper bounds on error and excess-rate exponents using a general converse approach based on information spectrum methods and type class analysis.
• Proposes a class of variable-rate Slepian-Wolf codes that assign the same rate to all source blocks of the same type class, enabling tractable achievability analysis.
• Uses random binning with type-dependent rates to derive lower bounds on the error exponent and excess-rate exponent, generalizing prior average-rate coding results.
• Applies variational methods and divergence minimization techniques, including a variant of Minkowski’s inequality and a parametric divergence function $ D(Q_\alpha \| P_1) $, to derive tight exponent bounds.
• Employs iterative algorithms based on alternating minimization to compute both the upper and lower bounds on the optimal rate functions and their corresponding excess-rate exponents.
• Establishes continuity and monotonicity properties of the divergence $ D(Q_\alpha \| P_1) $ as a function of $ \alpha $, which are critical for convergence and optimization.

Experimental results

Research questions

• RQ1 What is the fundamental trade-off between error exponent and excess-rate exponent in variable-rate Slepian-Wolf coding?
• RQ2 How can the minimal rate function per type class be determined to achieve a given target error exponent while minimizing excess-rate probability?
• RQ3 Can a type-based coding scheme achieve a better balance between reliability and rate overflow than fixed-rate or average-rate coding?
• RQ4 What are the tightest achievable upper and lower bounds on the error and excess-rate exponents for variable-rate Slepian-Wolf codes?
• RQ5 How can the optimal rate functions and their corresponding excess-rate exponents be computed efficiently?

Key findings

• The paper establishes that type-class uniform rate assignment achieves a balanced performance between fixed-rate and average-rate coding, offering a continuous trade-off between error exponent and excess-rate exponent.
• Upper and lower bounds on the error and excess-rate exponents are derived, with the bounds coinciding in the limit for certain classes of sources, indicating optimality of the proposed scheme.
• The minimal rate function per type class required to achieve a target error exponent is characterized via a convex optimization problem, solvable via iterative algorithms.
• The excess-rate exponent is shown to be strictly positive for the proposed code class, indicating that rate overflow is exponentially unlikely under the given rate allocation.
• The derived bounds generalize prior results on average-rate Slepian-Wolf coding by incorporating excess-rate constraints, which are critical for buffer-constrained systems.
• Iterative algorithms based on alternating minimization are proven effective for computing both the optimal rate functions and their associated excess-rate exponents with high accuracy.