[Paper Review] Equivalence of Two Proof Techniques for Non-Shannon-type Inequalities

Tarik Kaced|arXiv (Cornell University)|Feb 13, 2013

Computability, Logic, AI Algorithms7 references4 citations

TL;DR

This paper establishes the equivalence of two major proof techniques for deriving non-Shannon-type information inequalities: Zhang-Yeung's copy/paste lemma and Makarychev et al.'s coding lemma based on Ahlswede-Körner results. It proves that both techniques are equally powerful when applied to balanced inequalities—those where the sum of coefficients for each variable is zero—demonstrating that any inequality provable via one method can be reformulated and proven via the other, provided balancing is used as a preprocessing step.

ABSTRACT

We compare two different techniques for proving non-Shannon-type information inequalities. The first one is the original Zhang-Yeung's method, commonly referred to as the copy/pasting lemma/trick. The copy lemma was used to derive the first conditional and unconditional non-Shannon-type inequalities. The second technique first appeared in Makarychev et al paper [7] and is based on a coding lemma from Ahlswede and Körner works. We first emphasize the importance of balanced inequalities and provide a simpler proof of a theorem of Chan's for the case of Shannon-type inequalities. We compare the power of various proof systems based on a single technique.

Motivation & Objective

• To compare the relative power of two dominant proof techniques for non-Shannon-type information inequalities: Zhang-Yeung's copy/paste lemma and Makarychev et al.'s coding lemma.
• To investigate the role of balanced inequalities—where the sum of coefficients for each variable is zero—in enabling equivalence between the two proof techniques.
• To show that both techniques are equally powerful when applied to balanced inequalities, thereby unifying their theoretical foundations.
• To provide a simpler proof of Chan’s result on balanced inequalities for the special case of Shannon-type inequalities.
• To support the integration of both techniques into automated information inequality provers by demonstrating their interchangeability under balancing.

Proposed method

• Introduces and formalizes the concept of balanced inequalities, where the total coefficient sum for each variable across all terms is zero.
• Uses Chan’s theorem to show that any valid information inequality can be transformed into an equivalent balanced inequality by subtracting conditional entropy terms.
• Applies the copy/paste lemma (Rule ZY) to derive new inequalities by introducing auxiliary variables through copying.
• Applies the coding lemma (Rule MMRV) from Makarychev et al., which leverages entropy sub-achievable vector properties to derive stronger inequalities.
• Demonstrates that Rule ZY can simulate Rule MMRV and vice versa when working with balanced inequalities, via algebraic rewriting and coefficient balancing.
• Establishes that the inference power of both systems is equivalent modulo balancing, using a formal equivalence theorem (Theorem 4).

Experimental results

Research questions

• RQ1 Are Zhang-Yeung’s copy/paste lemma and Makarychev et al.’s coding lemma equally powerful for proving non-Shannon-type inequalities?
• RQ2 Can every inequality provable via one technique be reformulated and proven using the other, assuming balanced inequalities are used?
• RQ3 What is the role of balanced inequalities in unifying different proof techniques for non-Shannon-type inequalities?
• RQ4 Is there a formal equivalence between the two proof systems when restricted to balanced inequalities?
• RQ5 Can automated information inequality provers safely use either technique interchangeably if balancing is applied?

Key findings

• The two proof techniques—Zhang-Yeung’s copy/paste lemma and Makarychev et al.’s coding lemma—are formally equivalent in their inferential power when applied to balanced inequalities.
• Any inequality provable via the copy/paste lemma (Rule ZY) can be re-derived using the coding lemma (Rule MMRV) if the input inequality is balanced for the relevant variable.
• Conversely, any inequality provable via the coding lemma can be re-derived using the copy/paste lemma after balancing the input inequality.
• Balancing inequalities is not only sufficient but also necessary for achieving equivalence between the two techniques, as shown by the equivalence theorem (Theorem 4).
• The balanced counterpart of a non-Shannon-type inequality may not be Shannon-type, indicating that balancing extends the scope of provable inequalities beyond the Shannon-type class.
• Balanced inequalities are particularly valuable in automated provers because they are stronger than their unbalanced counterparts and allow seamless switching between proof techniques.