[Paper Review] A weight-distribution bound for entropy extractors using linear binary codes

Alessio Meneghetti, Massimiliano Sala+1 more|arXiv (Cornell University)|May 12, 2014

Computability, Logic, AI Algorithms1 references3 citations

TL;DR

This paper introduces a new bound on the total variation distance (TVD) of entropy extractor outputs using linear binary codes, leveraging the full weight distribution of the code instead of just the minimum distance. The key contribution is a tighter entropy rate lower bound that improves upon prior methods by incorporating codeword weight distribution, enabling better selection of extractors for higher-quality randomness from biased sources.

ABSTRACT

We consider a bound on the bias reduction of a random number generator by processing based on binary linear codes. We introduce a new bound on the total variation distance of the processed output based on the weight distribution of the code generated by the chosen binary matrix. Starting from this result we show a lower bound for the entropy rate of the output of linear binary extractors.

Motivation & Objective

• To improve the theoretical analysis of entropy extractors by moving beyond minimum distance to use the full weight distribution of linear codes.
• To provide a tighter bound on the total variation distance (TVD) between the extractor output and uniform distribution.
• To derive a more accurate lower bound on the entropy rate of the output of linear binary extractors.
• To enable better selection of generator matrices for extractors by evaluating performance based on full weight distribution, not just minimum distance.
• To compare the new bound with existing minimum-distance-based bounds and demonstrate its superiority in specific code families like Reed-Muller codes.

Proposed method

• The method introduces a new TVD bound (Theorem 3) that depends on the entire weight distribution of the linear code generated by the binary generator matrix.
• The bound is derived by analyzing the probability mass function of the extractor output and relating it to the weight distribution of codewords.
• The core equation is δ = ∑_{l=d}^{n} A_l ε_x^l, where A_l is the number of codewords of weight l and ε_x is the input bias.
• The entropy rate lower bound is derived using Sason’s theorem (Theorem 4), which relates TVD to Shannon entropy difference.
• The method applies this bound to compare extractors using Reed-Muller codes of different sizes, showing improved performance when full weight distribution is considered.
• The approach replaces the prior reliance on minimum distance d with a weighted sum over all codeword weights, improving accuracy in entropy estimation.

Experimental results

Research questions

• RQ1 Can the total variation distance of a linear binary entropy extractor be bounded more tightly by using the full weight distribution of the code instead of just the minimum distance?
• RQ2 How does the entropy rate of the extractor output compare when using a weight-distribution-based TVD bound versus a minimum-distance-based bound?
• RQ3 In what cases does incorporating the full weight distribution lead to a significant improvement in entropy rate estimation?
• RQ4 Can the new bound be used to guide the selection of optimal generator matrices for entropy extractors?
• RQ5 How do the bounds perform on specific code families such as Reed-Muller codes?

Key findings

• The new TVD bound (Theorem 3) is tighter than the minimum-distance-based bound (13) because it accounts for the entire weight distribution of the code, not just the minimum distance.
• For codes where codewords have weights greater than d, the new bound yields a significantly lower δ, leading to a better entropy rate lower bound.
• The entropy rate lower bound (Corollary 2) is expressed as H(Y) ≥ 1 - (δ/2)log_{2^k}(2^k - 1) - h(δ/2), with δ = ∑_{l=d}^{n} A_l ε_x^l.
• When all codewords have weight d, the new bound reduces to the prior minimum-distance-based bound δ = 2^k ε_x^d, confirming consistency.
• Experimental results on Reed-Muller [16,11] and [256,163] codes show that the weight-distribution-based bound provides a better (higher) entropy rate estimate than the minimum-distance-only bound.
• The results demonstrate that knowledge of the full weight distribution enables better extractor design and performance prediction, especially for codes with diverse codeword weights.