[Paper Review] Outer Bounds for Multiterminal Source Coding via a Strong Data Processing Inequality

Thomas A. Courtade|arXiv (Cornell University)|Feb 14, 2013

Wireless Communication Security Techniques8 citations

TL;DR

This paper introduces a novel outer bound for the multiterminal source coding problem using a strong data processing inequality, avoiding auxiliary random variables by leveraging the correlation measure $ s^* $, which quantifies how information degrades through a channel. The key contribution is a sum-rate outer bound that couples individual rate-distortion functions with $ s^*(X;Y) $ and $ s^*(Y;X) $, providing a tight, non-parameterized bound applicable to general sources and distortion measures.

ABSTRACT

An intuitive outer bound for the multiterminal source coding problem is given. The proposed bound explicitly couples the rate distortion functions for each source and correlation measures which derive from a "strong" data processing inequality. Unlike many standard outer bounds, the proposed bound is not parameterized by a continuous family of auxiliary random variables, but instead only requires maximizing two ratios of divergences which do not depend on the distortion functions under consideration.

Motivation & Objective

• To develop a general, non-parameterized outer bound for the multiterminal source coding problem that avoids the need for auxiliary random variables.
• To extend the applicability of strong data processing inequalities to multiterminal source coding beyond Gaussian settings.
• To provide a unified framework for deriving sum-rate outer bounds using $ s^*(X;Y) $, a measure of correlation derived from relative entropy.
• To offer a tighter, intuitive outer bound that captures the trade-off between individual rates and sum-rate constraints without relying on known converse results.

Proposed method

• The method uses a strong data processing inequality (SDPI) to relate the mutual information between sources and their compressed representations.
• It defines $ s^*(X;Y) = \sup_{Q_X \neq P_X} \frac{D(Q_Y \| P_Y)}{D(Q_X \| P_X)} $, where $ Q_Y $ is the output marginal under $ Q_X P_{Y|X} $, quantifying correlation strength.
• The outer bound is derived by applying the SDPI to the joint distribution of source sequences and their compressed versions, leveraging tensorization of $ s^* $ over i.i.d. sequences.
• The bound is expressed as $ \sum_{i} s^*(Y_i; X) R_i \geq \mathbb{R}_X(D_X) $, linking individual rates to the rate-distortion function.
• The approach avoids single-letterization by directly working with the $ s^* $ measure, enabling bounds without auxiliary variables.
• An alternate proof uses a convexity argument on the mutual information terms to complete the outer bound derivation.

Experimental results

Research questions

• RQ1 Can a strong data processing inequality be used to derive a non-parameterized outer bound for multiterminal source coding?
• RQ2 How can the correlation between sources be quantified in a way that directly informs rate-distortion trade-offs?
• RQ3 Can the sum-rate outer bound be derived without relying on auxiliary random variables or single-letter characterizations?
• RQ4 What is the role of $ s^*(X;Y) $ in capturing the fundamental limits of distributed source coding?
• RQ5 How does the proposed bound compare to known bounds like the cooperative lower bound or the Berger-Tung region?

Key findings

• The proposed outer bound is non-parameterized and depends only on $ s^*(X;Y) $ and $ s^*(Y;X) $, avoiding auxiliary random variables.
• For Gaussian sources, the bound recovers known sum-rate constraints and provides a tight approximation, especially when $ \rho $ is small.
• The bound generalizes the low-resolution approximation $ R_X + \rho^2 R_Y \geq \frac{1}{2}\log(1/D_X) $ to arbitrary sources via $ s^* $.
• The method yields a valid outer bound for the CEO problem: $ \sum_{i=1}^k s^*(Y_i; X) R_i \geq \mathbb{R}_X(D_X) $.
• An alternate proof of Zhao's common randomness capacity bound is derived without single-letterization, showing $ \frac{C(R)}{R} \leq \frac{1}{1 - s^*(X;Y)} $.
• The bound is tight in the sense that it captures the fundamental trade-off between compression rates and source correlation, especially when $ s^* $ is small.