[Paper Review] Moments of Sums of Independent and Identically Distributed Random Variables

Daniel M. Packwood|arXiv (Cornell University)|May 31, 2011

Stochastic processes and statistical mechanics7 references3 citations

TL;DR

This paper presents a systematic analytic method for computing the moments of sums of independent and identically distributed (i.i.d.) random variables using combinatorial formulas. It derives explicit expressions for the pth moment of $ S_n = \sum_{i=1}^n X_i $, enabling new proofs of the central limit theorem and generalized laws of large numbers, particularly showing $ S_n/n \Rightarrow H_\mu $, a unit-step distribution at $ \mu $, under finite moment conditions.

ABSTRACT

We present an analytic method for computing the moments of a sum of independent and identically distributed random variables. The limiting behavior of these sums is very important to statistical theory, and the moment expressions that we derive allow for it to be studied relatively easily. We show this by presenting a new proof of the central limit theorem and several other convergence results.

Motivation & Objective

• To develop a systematic method for computing the moments of sums of i.i.d. random variables, which are otherwise lacking in closed-form equality.
• To apply the moment formulas to derive new asymptotic results, including convergence in distribution of $ S_n/n $ to a unit-step distribution.
• To generalize the weak and strong laws of large numbers for i.i.d. sequences with finite moments, particularly for symmetric and asymmetric distributions.
• To provide a rigorous analytical foundation for the limiting behavior of normalized sums, especially the concentration of $ S_n/n $ into a point mass at $ \mu $.
• To demonstrate the utility of moment-based analysis in proving classical limit theorems, such as the central limit theorem, via explicit moment expansions.

Proposed method

• The method uses combinatorial enumeration to express the pth moment $ E(S_n^p) $ as a sum over all integer partitions of $ p $, where each term corresponds to a product of moments of $ X $.
• Each term in the moment expansion is weighted by a multinomial coefficient derived from the number of ways to assign indices to variables with repeated powers.
• The coefficient $ a_i $ in Equation (2) accounts for the number of distinct permutations of indices corresponding to a given power partition, incorporating factorials of multiplicities and falling factorials of $ n $.
• The formula is derived by expanding $ E(S_n^p) $ as a sum over all index combinations and grouping terms by their moment structure, leveraging independence and identical distribution.
• The method systematically handles all combinations of powers $ p_1, \dots, p_m $ summing to $ p $, with $ h $ distinct values and multiplicities $ l_1, \dots, l_h $.
• The approach ensures all moments are finite if and only if $ E(X^\alpha) < \infty $ for all $ \alpha \leq p $, guaranteeing convergence of the expressions.

Experimental results

Research questions

• RQ1 Can a general, systematic method be developed to compute the moments of sums of i.i.d. random variables, beyond existing inequalities?
• RQ2 Does the moment-based approach allow for new, direct proofs of classical limit theorems such as the central limit theorem?
• RQ3 What is the limiting distribution of $ S_n/n $ when the underlying distribution is asymmetric, and can it be characterized precisely?
• RQ4 Can the weak and strong laws of large numbers be generalized to higher-order moments, and under what conditions do they hold?
• RQ5 How do the asymptotic behaviors of $ E(S_n^p/n^p) $ differ between symmetric and asymmetric distributions?

Key findings

• The pth moment of $ S_n $ is given by a finite sum over all integer partitions of $ p $, with coefficients derived from multinomial and factorial combinatorics, as formalized in Equations (1) and (2).
• For the third moment, the formula yields $ E(S_n^3) = nE(X^3) + 3n(n-1)E(X^2)E(X) + n(n-1)(n-2)E(X)^3 $, confirming the combinatorial structure.
• As $ n \to \infty $, $ E(S_n^p / n^p) \to \mu^p $ for any $ p \geq 1 $, provided all moments of $ X $ up to order $ p $ are finite.
• The normalized sum $ S_n/n $ converges in distribution to a unit-step distribution $ H_\mu $, where $ P(H_\mu < h) = 0 $ for $ h < \mu $, and 1 otherwise.
• A generalized weak law of large numbers holds: $ \lim_{n \to \infty} P(|S_n^p / n^p - \mu^p| > \epsilon) = 0 $ for all $ \epsilon > 0 $, under finite moment conditions.
• For symmetric distributions with $ \mu = 0 $, the strong law generalization holds: $ S_n^p / n^p \to 0 $ almost surely, as shown via Borel-Cantelli lemma and moment decay.