[Paper Review] The myth of equidistribution for high-dimensional simulation

Richard P. Brent|arXiv (Cornell University)|May 8, 2010

Chaos-based Image/Signal Encryption8 references4 citations

TL;DR

This paper challenges the assumption that $(d,w)$-equidistribution is a desirable property for high-dimensional pseudo-random number generators (PRNGs). It argues that equidistribution is neither necessary nor sufficient for good simulation quality, as it depends on periodicity and does not reflect real-world usage where only short segments of the sequence are typically used. The key contribution is the refutation of equidistribution as a meaningful figure of merit for modern long-period PRNGs.

ABSTRACT

A pseudo-random number generator (RNG) might be used to generate w-bit random samples in d dimensions if the number of state bits is at least dw. Some RNGs perform better than others and the concept of equidistribution has been introduced in the literature in order to rank different RNGs. We define what it means for a RNG to be (d,w)-equidistributed, and then argue that (d,w)-equidistribution is not necessarily a desirable property.

Motivation & Objective

• To challenge the widely held belief that $(d,w)$-equidistribution is a reliable indicator of PRNG quality in high-dimensional simulations.
• To demonstrate that equidistribution is irrelevant for modern long-period PRNGs, especially when only short segments of the sequence are used.
• To argue that equidistribution testing, while computationally efficient, does not correlate with actual simulation performance.
• To re-evaluate existing figures of merit like $\Delta$ and $W$, showing they are misleading when comparing generators with different periods.
• To emphasize that equidistribution is independent of sequence ordering and thus fails to capture temporal or structural randomness needed in practice.

Proposed method

• Defining $(d,w)$-equidistribution as a property where each $d$-dimensional hypercube of side $2^{-w}$ contains exactly the same number of points over a full period.
• Using linear algebra over $\mathbb{F}_2$ to test equidistribution via matrix rank conditions for linear generators over $\mathbb{F}_2$.
• Analyzing the probability of equidistribution in a periodic sequence of length $N=2^n$, showing it is astronomically small for large $n$.
• Comparing equidistribution to actual simulation needs by highlighting that real applications use only $M \ll N$ points, not the full period.
• Reassessing figures of merit such as $\Delta = \max_d \delta_d$ and $W = \sum_d w_d$, arguing they are invalid for cross-period comparisons.
• Illustrating with RANDU that even widely used generators fail equidistribution for $d \geq 3$, $w \geq 4$, despite being periodic and deterministic.

Experimental results

Research questions

• RQ1 Is $(d,w)$-equidistribution a necessary or sufficient condition for a high-dimensional PRNG to be suitable for simulation?
• RQ2 Why is equidistribution considered a desirable property despite its poor alignment with practical simulation usage?
• RQ3 How does the probability of achieving $(d,w)$-equidistribution scale with the period length $N=2^n$?
• RQ4 Can equidistribution testing over a full period be meaningfully applied to real-world simulations that use only short segments of the sequence?
• RQ5 Are existing figures of merit like $\Delta$ and $W$ valid for comparing PRNGs with different periods?

Key findings

• The probability of a random periodic sequence of length $N=2^n$ being $(d,w)$-equidistributed is approximately $\frac{\sqrt{2\pi N}}{\exp(N)}$, which is negligible for large $n$, such as $n=4096$.
• Even if a generator is $(d,w)$-equidistributed, this property is invariant under reordering of the sequence, making it irrelevant for temporal or structural randomness.
• For $dw = n$, the number of ways to achieve equidistribution is $N!$, but the total number of possible sequences is $N^N$, so equidistribution is extremely unlikely in practice.
• Modern long-period PRNGs like the Mersenne Twister or Xorgens are often $(d,w)$-equidistributed for $dw \ll n$, but this does not imply better simulation quality.
• Figures of merit like $\Delta$ and $W$ are misleading when comparing generators with different periods, as they do not account for the fact that only short segments are used in practice.
• Equidistribution is not a reliable test of randomness because a perfectly random source is not periodic, and periodicity itself undermines the assumption of independence in simulations.