[Paper Review] The Linear Information Coupling Problems

Shao‐Lun Huang, Lizhong Zheng|arXiv (Cornell University)|Jun 11, 2014

Wireless Communication Security Techniques15 references5 citations

TL;DR

This paper introduces a geometric framework for solving multi-terminal information theory problems by approximating the space of probability distributions as a Euclidean space under local perturbations. By leveraging Kullback-Leibler divergence linearization via the squared Euclidean norm, the authors reduce complex information coupling problems to linear algebra, enabling systematic single-letterization and revealing coherent combining gains in multiple access channels and trade-offs in broadcast channels.

ABSTRACT

Many network information theory problems face the similar difficulty of single-letterization. We argue that this is due to the lack of a geometric structure on the space of probability distribution. In this paper, we develop such a structure by assuming that the distributions of interest are close to each other. Under this assumption, the K-L divergence is reduced to the squared Euclidean metric in an Euclidean space. In addition, we construct the notion of coordinate and inner product, which will facilitate solving communication problems. We will present the application of this approach to the point-to-point channel, general broadcast channel, and the multiple access channel (MAC) with the common source. It can be shown that with this approach, information theory problems, such as the single-letterization, can be reduced to some linear algebra problems. Moreover, we show that for the general broadcast channel, transmitting the common message to receivers can be formulated as the trade-off between linear systems. We also provide an example to visualize this trade-off in a geometric way. Finally, for the MAC with the common source, we observe a coherent combining gain due to the cooperation between transmitters, and this gain can be quantified by applying our technique.

Motivation & Objective

• To address the fundamental challenge of single-letterization in multi-terminal information theory by introducing a geometric structure on probability distributions.
• To model small perturbations in input distributions as local linear approximations in a Euclidean space via K-L divergence linearization.
• To unify the treatment of point-to-point, broadcast, and multiple access channels under a common framework based on linear algebra.
• To demonstrate that coherent combining gains in multiple access channels and trade-offs in broadcast channels emerge naturally from this geometric formulation.

Proposed method

• Assume input and output distributions are locally close, reducing Kullback-Leibler divergence to squared Euclidean distance in a tangent space.
• Define coordinate systems and inner products on the space of distributions to enable linear algebraic manipulation.
• Formulate the linear information coupling problem as maximizing mutual information under constraints on information modulation and distribution deviation.
• Use singular value decomposition of divergence transition matrices (DTM) to analyze channel capacity and information flow.
• Apply tensor product structures to extend results from single to multi-user channels, preserving singular value invariance across block lengths.
• Prove that optimal solutions correspond to singular vectors of the DTM, with orthogonality constraints ensuring maximal information transfer.

Experimental results

Research questions

• RQ1 How can single-letterization be systematically achieved in general multi-terminal communication problems?
• RQ2 What geometric structure emerges in the space of probability distributions under local perturbations?
• RQ3 How does the coherent combining gain in multiple access channels arise from the interaction of transmitters?
• RQ4 What is the nature of the trade-off between common and private messages in a broadcast channel under this framework?
• RQ5 Can the solution to the linear information coupling problem be integrated to recover global capacity-achieving schemes?

Key findings

• The linear information coupling problem reduces to a single-letter optimization problem under local perturbation assumptions, enabling systematic single-letterization.
• The second-largest singular value of the divergence transition matrix (DTM) remains invariant across block lengths, implying stable performance scaling.
• For the multiple access channel with a common source, a coherent combining gain emerges due to constructive interference, quantified via the DTM's singular vectors.
• In the broadcast channel, transmitting a common message corresponds to a linear system trade-off, visualized geometrically through singular vector alignment.
• The optimal input distribution perturbation lies orthogonal to the square-root of the input distribution, ensuring maximal information transfer under power constraints.
• The method recovers known capacity-achieving schemes by integrating solutions of local linear coupling problems, demonstrating its global consistency.