[Paper Review] On Optimal Causal Coding of Partially Observed Markov Sources in Single and Multi-Terminal Settings

Serdar Yüksel|arXiv (Cornell University)|Oct 22, 2010

Wireless Communication Security Techniques38 references4 citations

TL;DR

This paper establishes structural results for optimal causal coding of partially observed Markov sources in single- and multi-terminal settings, extending Witsenhausen’s and Walrand-Varaiya’s frameworks to general state spaces. It proves that for i.i.d. sources, optimal decentralized causal coding is memoryless, and presents a counterexample to a natural separation conjecture in Markov settings.

ABSTRACT

The optimal causal coding of a partially observed Markov process is studied, where the cost to be minimized is a bounded, non-negative, additive, measurable single-letter function of the source and the receiver output. A structural result is obtained extending Witsenhausen's and Walrand-Varaiya's structural results on optimal real-time coders to a partially observed setting. The decentralized (multi-terminal) setup is also considered. For the case where the source is an i.i.d. process, it is shown that the optimal decentralized causal coding of correlated observations problem admits a solution which is memoryless. For Markov sources, a counterexample to a natural separation conjecture is presented.

Motivation & Objective

• To extend structural results on optimal causal coders from fully observed to partially observed Markov processes with general state spaces.
• To analyze decentralized (multi-terminal) causal coding under partial observation, particularly with feedback and finite-capacity channels.
• To investigate whether a separation principle—where estimation and coding can be designed independently—holds in partially observed Markov settings.
• To determine conditions under which optimal coding policies are memoryless, especially for i.i.d. sources.
• To resolve open questions about the structure of optimal policies in team decision problems with asymmetric information and delayed feedback.

Proposed method

• Formalizes a causal coding problem with a partially observed Markov source, where the encoder observes noisy versions of the state and must quantize in real time.
• Introduces a composite quantization policy that combines local information and common (feedback) information, using a Markov Decision Process (MDP) framework.
• Uses a team decision theory approach to decompose the policy into a quantizer selection policy and a quantizer action, enabling structural analysis.
• Applies dynamic programming and sufficient statistics to show that optimal policies depend only on the posterior distribution of the state given past quantized outputs.
• Derives a policy structure that depends only on the common information (past quantized outputs) and local observations, proving equivalence to a restricted policy class.
• Employs a backward induction argument over time stages to show that optimal quantizers at each time depend only on the current posterior distribution and past quantized actions.

Experimental results

Research questions

• RQ1 Can structural results for optimal causal coders—previously known for fully observed systems—be extended to partially observed Markov processes with general state spaces?
• RQ2 In a decentralized multi-terminal setting with asymmetric information and feedback, what is the optimal structure of causal coding policies?
• RQ3 Is the optimal causal coding policy memoryless when the source is i.i.d., even under partial observation?
• RQ4 Does a separation principle hold in causal coding of partially observed Markov sources, where estimation and coding can be separated?
• RQ5 What is the minimal sufficient statistic for the optimal coding policy in a causal, decentralized setting with noisy observations and feedback?

Key findings

• An optimal causal coder for a partially observed Markov source depends only on the posterior distribution of the state given past quantized outputs and local observations.
• For i.i.d. sources, the optimal decentralized causal coding policy is memoryless, meaning the quantizer at each time depends only on the current local observation and past quantized outputs.
• A counterexample is constructed showing that the natural separation conjecture—where estimation and coding can be separated—does not hold for Markov sources under causal coding.
• The optimal policy structure is shown to be equivalent to a restricted class of policies that depend only on the common information (past quantized outputs) and local observations.
• The existence of an optimal policy is guaranteed due to the finiteness of the policy space in the restricted class, enabling a constructive solution via dynamic programming.
• The cost function at each time stage can be expressed as a measurable function of the posterior distribution of the observations given past quantized outputs, confirming the sufficiency of this statistic.