[Paper Review] Second-Order Coding Rates for Conditional Rate-Distortion

Sy-Quoc Le, Vincent Y. F. Tan+1 more|arXiv (Cornell University)|Oct 10, 2014

Wireless Communication Security Techniques26 references4 citations

TL;DR

This paper establishes second-order coding rates for lossy source coding with side information available at both encoder and decoder, using non-asymptotic bounds for discrete memoryless, Gaussian, and Markov sources. The key result is that for Gaussian sources with Gaussian side information, the dispersion is exactly 1/2 nats squared per symbol, independent of side information variance, matching the rate-distortion function without side information.

ABSTRACT

This paper characterizes the second-order coding rates for lossy source coding with side information available at both the encoder and the decoder. We first provide non-asymptotic bounds for this problem and then specialize the non-asymptotic bounds for three different scenarios: discrete memoryless sources, Gaussian sources, and Markov sources. We obtain the second-order coding rates for these settings. It is interesting to observe that the second-order coding rate for Gaussian source coding with Gaussian side information available at both the encoder and the decoder is the same as that for Gaussian source coding without side information. Furthermore, regardless of the variance of the side information, the dispersion is $1/2$ nats squared per source symbol.

Motivation & Objective

• To characterize finite-blocklength performance limits for lossy source coding when side information is available at both encoder and decoder.
• To derive non-asymptotic bounds that capture the trade-off between rate, distortion, and blocklength in this setting.
• To establish second-order coding rates for discrete memoryless, Gaussian, and Markov sources under conditional rate-distortion constraints.
• To investigate how side information affects the dispersion and second-order asymptotics in lossy source coding.
• To demonstrate that for Gaussian sources, the dispersion remains 1/2 nats² per symbol regardless of side information variance.

Proposed method

• Derives a non-asymptotic achievability bound using information spectrum methods and the Gallager function for the conditional rate-distortion problem.
• Applies the Berry-Esseen theorem and concentration inequalities to approximate the finite-blocklength behavior of the coding rate.
• Uses the central limit theorem and Taylor expansion to analyze the second-order term in the asymptotic expansion of the rate-distortion function.
• Establishes the second-order coding rate by analyzing the variance (dispersion) of the conditional information density $ j_{X|S}(x|s) $.
• Applies a Markov chain property to show that the conditional information density sequence forms a Markov process under certain conditions.
• Employs a refined large deviation bound with a logarithmic correction term to tighten the error probability analysis.

Experimental results

Research questions

• RQ1 What is the second-order coding rate for discrete memoryless sources with side information at both encoder and decoder?
• RQ2 How does the presence of side information affect the dispersion in Gaussian source coding with squared-error distortion?
• RQ3 Does the variance of the side information impact the second-order asymptotics in Gaussian settings?
• RQ4 What is the second-order coding rate for Markov sources when the joint source-side information sequence forms a time-homogeneous Markov chain?
• RQ5 Can the second-order coding rate for conditional rate-distortion be characterized using non-asymptotic bounds that account for finite blocklength effects?

Key findings

• For discrete memoryless sources with finite alphabets and Hamming distortion, the second-order coding rate is derived explicitly, extending known results to the conditional case.
• For Gaussian sources with Gaussian side information and squared-error distortion, the second-order coding rate is characterized with a dispersion of exactly 1/2 nats squared per symbol.
• The dispersion for the Gaussian case is independent of the variance of the side information, a counterintuitive result that holds despite the presence of side information.
• For Markov sources where the joint source-side information sequence forms a time-homogeneous Markov chain, the second-order coding rate is established via the conditional information density variance.
• The second-order coding rate for the Gaussian case matches that of the no-side-information case, indicating no improvement in dispersion from having side information at the encoder.
• The non-asymptotic bounds are tight and enable the derivation of second-order asymptotics with error probability convergence rates of order $ O(rac{ ext{log } n}{n}) $.