[Paper Review] A Linear Programming Inequality with Applications to Concentration of Measure

Leonid Kontorovich|ArXiv.org|Oct 24, 2006

Markov Chains and Monte Carlo Methods6 references3 citations

TL;DR

This paper presents a novel linear programming inequality that bounds the dual norm of a function in terms of a recursively defined functional, enabling tighter concentration of measure bounds for dependent random variables under weighted Hamming metrics. The key contribution is a generalization of prior martingale difference inequalities to weighted settings, leading to improved exponential tail bounds for Lipschitz functions on product spaces with arbitrary positive weights.

ABSTRACT

We prove an elementary yet useful inequality bounding the maximal value of certain linear programs. This leads directly to a bound on the martingale difference for arbitrarily dependent random variables, providing a generalization of some recent concentration of measure results. The linear programming inequality may be of independent interest.

Motivation & Objective

• To derive a general linear programming inequality that bounds the dual norm of functions on finite product spaces with respect to weighted Hamming metrics.
• To extend prior concentration of measure results for martingale differences to arbitrary positive weight vectors, generalizing unweighted results from previous work.
• To provide a unified framework for bounding the variance of Lipschitz functions under dependence structures via a recursive functional norm.
• To establish a tight exponential tail bound for deviations of Lipschitz functions from their mean, applicable to arbitrarily dependent random variables.

Proposed method

• Introduces a recursive functional $\Psi_{w,n}(\kappa)$ that computes a weighted sum of non-negative parts of function values, recursively projecting over coordinates.
• Defines two key norms: the $\Phi_{w}$-norm as the supremum of inner products with 1-Lipschitz functions under the weighted Hamming metric, and the $\Psi_{w}$-norm as the maximum of $\Psi_{w,n}$ over sign flips of the function.
• Proves the inequality $\|\kappa\|_{\Phi,w} \leq \|\kappa\|_{\Psi,w}$ via induction, using properties of marginal projection and section operators on function spaces.
• Applies the inequality to derive a bound on the variance of martingale differences by relating the variance to the $\ell_2$-norm of a matrix $\Delta_n$ encoding conditional dependencies.
• Uses the matrix $\Delta_n$ with entries $\bar{\eta}_{ij}$, representing conditional dependence strengths, to express the variance bound in terms of $\|\Delta_n w\|_2^2$.
• Derives a concentration inequality via the bounded differences method, showing that the deviation probability decays exponentially in $t^2 / (\|f\|_{\text{Lip},w}^2 \|\Delta_n\|_2^2 \|w\|_2^2)$.

Experimental results

Research questions

• RQ1 Can a general linear programming inequality be established that bounds the dual norm of functions under weighted Hamming metrics, independent of the underlying probability measure?
• RQ2 How can the martingale difference variance be bounded in terms of a dependency structure encoded in a matrix $\Delta_n$ for arbitrarily dependent random variables?
• RQ3 To what extent does the new inequality improve upon existing concentration bounds for Lipschitz functions on product spaces with non-i.i.d. components?
• RQ4 Can the recursive functional $\Psi_{w,n}$ be used to derive tight, computable variance bounds for functions with arbitrary weights and dependence?
• RQ5 Does the generalization from unweighted to weighted Hamming metrics preserve the exponential tail decay in concentration inequalities?

Key findings

• The inequality $\|\kappa\|_{\Phi,w} \leq \|\kappa\|_{\Psi,w}$ holds for all $w \in \mathbb{R}_+^n$ and all $\kappa \in K_n$, providing a tight bound on the dual norm of functions under weighted Hamming metrics.
• The variance of the martingale difference sequence satisfies $\sum_{i=1}^n \bar{V}_i^2(f) \leq \|f\|_{\text{Lip},w}^2 \|\Delta_n w\|_2^2$, where $\Delta_n$ encodes conditional dependence strengths between variables.
• For any $f: \mathcal{S}^n \to \mathbb{R}$, the tail probability satisfies $\mathbf{P}\{ |f - \mathbf{E}f| > t \} \leq 2\exp\left(-\frac{t^2}{2\|f\|_{\text{Lip},w}^2 \|\Delta_n\|_2^2 \|w\|_2^2}\right)$, generalizing previous results to weighted settings.
• The bound is tight in the sense that it reduces to known unweighted results when $w_i \equiv 1$, and extends naturally to countable state spaces.
• The recursive structure of $\Psi_{w,n}$ allows for efficient computation of the norm and enables the derivation of variance bounds without requiring independence.
• The method provides a systematic way to incorporate arbitrary dependency structures into concentration inequalities via the matrix $\Delta_n$, which captures conditional expectations between variables.