[Paper Review] On the Bennett-Hoeffding inequality

Iosif Pinelis|arXiv (Cornell University)|Feb 24, 2009

Probability and Risk Models38 references4 citations

TL;DR

This paper refines the Bennett-Hoeffding inequality for sums of independent, bounded zero-mean random variables by incorporating truncated third moments and replacing the class of exponential functions with a broader class of generalized moment functions that are increasing, convex, and have convex second derivatives. The resulting bounds are tighter than existing ones, especially when skewness is low, and exhibit optimality properties. The method extends to martingales and supermartingales.

ABSTRACT

The well-known Bennett-Hoeffding bound for sums of independent random variables is refined, by taking into account truncated third moments, and at that also improved by using, instead of the class of all increasing exponential functions, the much larger class of all generalized moment functions f such that f and f" are increasing and convex. It is shown that the resulting bounds have certain optimality properties. Comparisons with related known bounds are given. The results can be extended in a standard manner to (the maximal functions of) (super)martingales.

Motivation & Objective

• To improve the classical Bennett-Hoeffding inequality by incorporating higher-order moment information beyond the variance.
• To develop a tighter upper bound on the tail probability P(S ≥ x) for sums of independent, bounded random variables.
• To replace the restrictive class of exponential functions with a larger class of generalized moment functions that preserve convexity and monotonicity.
• To establish optimality properties of the new bounds under fixed parameters including variance and skewness.
• To extend the refined bounds to (super)martingales via standard probabilistic techniques.

Proposed method

• Introduces a new class of functions f such that f and f'' are increasing and convex, generalizing exponential functions used in classical bounds.
• Derives an improved upper bound on the moment-generating function E[e^{λS}] by incorporating the truncated third moment β⁺₃ = Σ E[(Xᵢ)₃⁺].
• Constructs a new tail bound PU(x) = inf_{λ>0} exp(-λx) × PUexp(λ), where PUexp(λ) includes a correction term involving ε = β⁺₃ / (σ²y).
• Uses a variational approach to minimize the tail bound over λ > 0, leveraging the structure of the generalized moment functions.
• Applies the method to (super)martingales by extending the results through standard martingale techniques.
• Employs analytical and symbolic computation (via Mathematica) to verify positivity of key expressions in multiple parameter regimes, ensuring validity of the bounds.

Experimental results

Research questions

• RQ1 Can the Bennett-Hoeffding bound be improved by incorporating third-moment information beyond variance?
• RQ2 Does replacing exponential functions with a broader class of generalized moment functions yield tighter and more optimal tail bounds?
• RQ3 How do the new bounds behave asymptotically in the large deviation regime compared to classical bounds?
• RQ4 What is the optimality structure of the new bounds under fixed variance and skewness parameters?
• RQ5 Can the refined bounds be extended to (super)martingale processes with similar tightness?

Key findings

• The new bound PU(x) is strictly tighter than the classical Bennett-Hoeffding bound BH(x) for all x ≥ 0, with the improvement being significant when ε = β⁺₃ / (σ²y) ≪ 1.
• The bound PUexp(λ) is the exact upper bound on E[e^{λS}] when λ, y, σ², and ε are fixed, establishing its optimality in this parameter set.
• For large x, the new bounds recover the correct 1/x factor missing in classical exponential bounds, improving on the normal approximation by restoring the correct tail decay rate.
• The bound Ea(x) = inf_{t∈(0,x)} E[(|Z|−t)₃⁺]/(x−t)³ provides the best possible upper bound for P(|S| ≥ x) under Eaton-type comparison inequalities.
• The method is extendable to (super)martingales, preserving the tightness and optimality of the bounds.
• Symbolic verification using Mathematica confirms the positivity of key expressions across all parameter regimes, validating the analytical derivations.