[Paper Review] Proof of a Combinatorial Conjecture Coming from the PAC-Bayesian Machine Learning Theory

Malik Younsi|arXiv (Cornell University)|Sep 4, 2012

Computability, Logic, AI AlgorithmsComputer Science1 references17 citations

TL;DR

This paper proves a combinatorial conjecture from PAC-Bayesian machine learning theory, establishing that ξ₂(m) = ξ(m) + m for all natural numbers m. The proof leverages Abel’s binomial theorem and its multinomial generalization by Hurwitz, deriving closed-form expressions for ξ(m) and ξ₂(m) using factorial and binomial identities, which significantly simplify numerical computation and validate the conjecture.

ABSTRACT

We give a proof of a conjecture of A. Lacasse in his doctoral thesis which has applications in machine learning algorithms. The proof relies on some interesting binomial sums identities introduced by Abel (1839), and on their generalization to the multinomial case by Hurwitz (1902).

Motivation & Objective

• To resolve a long-standing combinatorial conjecture arising in PAC-Bayesian machine learning theory.
• To provide closed-form expressions for ξ(m) and ξ₂(m), which are central to generalization bounds in learning algorithms.
• To validate the conjecture ξ₂(m) = ξ(m) + m using established combinatorial identities.
• To offer computationally more efficient representations of ξ(m) and ξ₂(m) through factorial and binomial sum identities.

Proposed method

• Transform the original conjecture by scaling ξ(m) and ξ₂(m) via multiplication by m^m to obtain α(m) and β(m), respectively.
• Apply Abel’s binomial theorem in the form Aₘ(0,0;0,0) = ∑ₖ₌₀ᵐ (m choose k) k! m^{m−k}, which yields α(m) = ∑ⱼ₌₀ᵐ mʲ m!/j!.
• Use the multinomial generalization of Abel’s identity for n=3, x₁=x₂=x₃=0, p₁=p₂=p₃=0, to derive β(m) = ∑ⱼ₌₀ᵐ (m choose j) (j+1)! m^{m−j}.
• Verify that β(m) − α(m) = m^{m+1}, which implies ξ₂(m) = ξ(m) + m after scaling back by m^{−m}.
• Confirm the derived identities using references from the literature, specifically Hurwitz’s 1902 generalization of multinomial sums.
• Establish the equivalence of the conjecture to the identity β(m) − α(m) = m^{m+1}, which is proven via combinatorial identities.

Experimental results

Research questions

• RQ1 Does the conjecture ξ₂(m) = ξ(m) + m hold for all natural numbers m?
• RQ2 Can closed-form expressions be derived for ξ(m) and ξ₂(m) using known combinatorial identities?
• RQ3 What is the connection between the PAC-Bayesian framework and multinomial sum identities?
• RQ4 How can the original conjecture be reduced to a problem involving factorial and binomial sums?

Key findings

• The conjecture ξ₂(m) = ξ(m) + m is proven true for all m ∈ ℕ.
• A closed-form expression for ξ(m) is derived as ξ(m) = m^{−m} ∑ⱼ₌₀ᵐ mʲ m!/j!.
• A closed-form expression for ξ₂(m) is derived as ξ₂(m) = m^{−m} ∑ⱼ₌₀ᵐ (m choose j) (j+1)! m^{m−j}.
• The identity β(m) − α(m) = m^{m+1} is established, which directly implies the main conjecture after scaling.
• The results are validated through known identities from Abel (1839) and Hurwitz (1902), confirming the correctness of the derivations.
• The derived expressions are more numerically stable and efficient than direct summation of the original forms.