[Paper Review] Source Coding Problems with Conditionally Less Noisy Side Information

Roy Timo, Tobias J. Oechtering+1 more|arXiv (Cornell University)|Dec 11, 2012

Wireless Communication Security Techniques20 references4 citations

TL;DR

This paper establishes the optimality of Heegard and Berger's single-letter rate-distortion (RD) bound for a new class of source coding problems where one receiver's side information is conditionally less noisy than the other's. Using a novel single-letterization lemma based on Kramer's information-theoretic telescoping identity, the authors prove a converse that confirms the achievability bound is tight under this condition, extending prior results for physically degraded side information and generalizing to successive refinement settings.

ABSTRACT

A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for \emph{physically degraded} side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup in which the side information at one receiver is \emph{conditionally less noisy} than the side information at the other. The new setup includes degraded side information as a special case, and it is motivated by the literature on degraded and less noisy broadcast channels. Our key contribution is a converse proving the optimality of Heegard and Berger's achievability bound in a new setting. The converse rests upon a certain \emph{single-letterization} lemma, which we prove using an information theoretic telescoping identity {recently presented by Kramer}. We also generalise the above ideas to two different successive-refinement problems.

Motivation & Objective

• To close the gap in characterizing the rate-distortion function for multiple receiver source coding with arbitrarily correlated side information.
• To identify a new class of side information structures—conditionally less noisy side information—where Heegard and Berger's achievability bound is optimal.
• To generalize the converse technique to successive refinement problems with side information, including physically degraded and scalable side information setups.
• To develop a new single-letterization lemma based on Kramer's telescoping identity to enable the converse proof.

Proposed method

• Derives a single-letterization lemma that expresses the difference of two n-letter conditional mutual informations using a single-letter expression involving auxiliary random variables.
• Applies the lemma to a two-receiver source coding problem with side information, using a telescoping identity to simplify the converse chain of inequalities.
• Establishes a converse for the Heegard-Berger problem under the conditionally less noisy side information assumption, proving optimality of their achievability bound.
• Extends the method to two successive-refinement problems: one with physically degraded side information and one with scalable side information, deriving new converse bounds.
• Uses Fano’s inequality and continuity arguments to handle asymptotic limits as n → ∞ and ε → 0.
• Introduces auxiliary random variables W₁, W₂, W₃ with bounded alphabet sizes (≤ |𝒳|) to achieve single-letter characterization.

Experimental results

Research questions

• RQ1 Is Heegard and Berger’s single-letter achievability bound optimal for source coding with arbitrarily correlated side information?
• RQ2 Can the converse proof be extended beyond physically degraded side information to a broader class of side information structures?
• RQ3 Does the conditionally less noisy side information model allow for a tight converse that matches Heegard and Berger’s bound?
• RQ4 Can the proposed single-letterization lemma be applied to successive refinement problems with side information?
• RQ5 Can the telescoping identity technique be used to derive new converses in multiterminal source coding?

Key findings

• The converse proves that Heegard and Berger’s achievability bound is optimal for the conditionally less noisy side information model, resolving a long-standing open problem in this setting.
• The key technical contribution is a single-letterization lemma that enables the converse proof via Kramer’s information-theoretic telescoping identity.
• The converse is established under a deterministic distortion function at one receiver, with the bound holding in the limit as ε → 0 and n → ∞.
• The method generalizes to two successive-refinement problems: one with physically degraded side information and one with scalable side information, yielding new converse bounds.
• The auxiliary random variables W₁, W₂, W₃ introduced in the converse have alphabet sizes bounded by |𝒳|, ensuring single-letterizability.
• The proof relies on the assumption that (Y₁ ⪰ Y₂ | X̃₂), meaning Y₁ is conditionally less noisy than Y₂ given X̃₂, which generalizes the physically degraded case.