[Paper Review] Generalized symmetric mutual information applied for the channel capacity

Takuya Yamano|arXiv (Cornell University)|Feb 19, 2001

Advanced Wireless Communication Techniques1 references3 citations

TL;DR

This paper proposes a symmetric, generalized mutual information measure based on nonadditive entropy (Tsallis-type) to redefine channel capacity for the binary symmetric channel (BSC). By ensuring symmetry in input and output, it derives a generalized capacity $ C_q $ that can exceed Shannon’s capacity for $ q < 1 $, but may become negative for $ q > 1 $, highlighting a fundamental tension between symmetry and non-negativity in nonextensive information theory.

ABSTRACT

The channel capacity for the binary symmetric channel is investigated based on the symmetrized definition of the mutual information, which is arising from an attempt of extension of information content based on the nonadditivity. The negative capacity can emerge as an avoidable consequence for the generalization of the concept of the information entropy when $q &gt;1$.

Motivation & Objective

• To resolve asymmetry in generalized mutual information definitions within nonadditive information theory.
• To investigate whether symmetric mutual information preserves the zero-capacity condition at $ e = 1/2 $, as in Shannon’s theory.
• To derive and analyze the generalized channel capacity $ C_q $ for the binary symmetric channel using the new symmetric mutual information definition.
• To examine the validity and interpretability of negative channel capacities in nonadditive information frameworks.

Proposed method

• Introduces a symmetric generalized mutual information $ I_q(X;Y) = H_q(X) + H_q(Y) - H_q(X,Y) $, derived from a modified Tsallis entropy with normalization.
• Defines the nonadditive joint entropy $ H_q(X,Y) $ using the relation $ H_q(X,Y) = H_q(Y) + H_q(X|Y) + (q-1)H_q(Y)H_q(X|Y) $, ensuring consistency with nonadditive structure.
• Applies the generalized mutual information to the binary symmetric channel (BSC), with input probabilities $ p_0, p_1 $, and error probability $ e $, computing joint probabilities $ p_{xy} $ explicitly.
• Derives the generalized capacity $ C_q = \max_{p(x)} I_q(X;Y) $, showing maximum occurs at $ p_0 = p_1 = 1/2 $ for certain $ q $ and $ e $.
• Expresses $ I_q(X;Y) $ in closed form using $ f(p_0,e) = [(1-2e)p_0 + e]^q + [(2e-1)p_0 + 1 - e]^q $, enabling analytical evaluation of $ C_q $.
• Evaluates $ C_q $ analytically and numerically, comparing it to Shannon’s capacity and analyzing its behavior across $ q $ and $ e $.

Experimental results

Research questions

• RQ1 Can a symmetric definition of generalized mutual information be consistently formulated in nonadditive information theory without violating key properties like zero capacity at $ e = 1/2 $?
• RQ2 What is the behavior of the generalized channel capacity $ C_q $ for the binary symmetric channel under symmetric mutual information, especially for $ q \neq 1 $?
• RQ3 Does the generalized capacity $ C_q $ exceed Shannon’s capacity for any $ q < 1 $, and under what noise conditions?
• RQ4 Why does the generalized capacity become negative for $ q > 1 $, and what does this imply for the physical interpretability of nonadditive information measures?
• RQ5 Is there a fundamental trade-off between symmetry in mutual information and non-negativity of channel capacity in generalized information theory?

Key findings

• The generalized channel capacity $ C_q $ exceeds Shannon’s capacity for intermediate noise levels when $ q < 1 $, indicating potential advantages in noisy regimes.
• For $ q > 1 $, the generalized capacity can become negative, which contradicts the physical intuition of capacity as a non-negative measure of information transmission.
• The maximum capacity is achieved at uniform input distribution $ p_0 = p_1 = 1/2 $ for certain ranges of $ q $ and $ e $, preserving symmetry.
• The derived expression for $ C_q = \frac{1}{q-1}\left(2^q - 1 - \frac{2^{q-1}}{e^q + (1-e)^q}\right) $ provides a closed-form analytical solution for BSC capacity under the new symmetric definition.
• The capacity does not vanish at $ e = 1/2 $ for $ q \neq 1 $, violating the standard requirement of zero capacity at maximum noise, unless $ q = 1 $.
• There exists a fundamental trade-off: enforcing symmetry in mutual information leads to negative capacity for $ q > 1 $, while preserving zero capacity at $ e = 1/2 $ requires sacrificing symmetry.