[Paper Review] Distribution of Euclidean Distances Between Randomly Distributed Gaussian Points in n-Space

Benjamin Thirey, Randal Hickman|arXiv (Cornell University)|Aug 10, 2015

Data Management and Algorithms8 references9 citations

TL;DR

This paper derives the exact probability distribution of Euclidean distances between pairs of points independently drawn from a standard normal distribution in k-dimensional space. Using a change of variables and properties of chi-distributions, it provides closed-form expressions for the raw and central moments of the distance distribution, revealing how distance variability and skewness evolve with increasing dimensionality—critical for assessing the reliability of nearest-neighbor and clustering algorithms in high-dimensional data.

ABSTRACT

The curse of dimensionality is a common phenomenon which affects analysis of datasets characterized by large numbers of variables associated with each point. Problematic scenarios of this type frequently arise in classification algorithms which are heavily dependent upon distances between points, such as nearest-neighbor and $k$-means clustering. Given that contributing variables follow Gaussian distributions, this research derives the probability distribution that describes the distances between randomly generated points in n-space. The theoretical results are extended to examine additional properties of the distribution as the dimension becomes arbitrarily large. With this distribution of distances between randomly generated points in arbitrarily large dimensions, one can then determine the significance of distance measurements between any collection of individual points.

Motivation & Objective

• To derive the probability distribution of Euclidean distances between two i.i.d. standard normal points in k-dimensional space.
• To quantify how distance distributions change as dimension k increases, especially in relation to the curse of dimensionality.
• To provide closed-form expressions for raw and central moments (up to fourth order) of the distance distribution for use in statistical inference.
• To support the evaluation of spurious associations in high-dimensional datasets by modeling baseline distance behavior under randomness.
• To offer a theoretical foundation for assessing the significance of distance-based classifications in machine learning, particularly in k-NN and k-means clustering.

Proposed method

• Transforms the joint distribution of two k-variate standard normal vectors into a radial distance distribution via change of coordinates.
• Derives the cumulative distribution function (CDF) of the distance by integrating over the region where |p - q| ≤ r.
• Obtains the probability density function (PDF) of the distance R by differentiating the CDF, resulting in a closed-form expression involving gamma functions.
• Computes the n-th raw moment of R using the formula E[R^n] = 2^{1-k} Γ((k+n)/2) / Γ(k/2) for n ≥ 0.
• Calculates the second, third, and fourth central moments explicitly using integrals involving R, μ, and the derived PDF.
• Derives expressions for skewness and kurtosis using the central moments, enabling characterization of the shape of the distance distribution in high dimensions.

Experimental results

Research questions

• RQ1 What is the exact probability distribution of Euclidean distances between two i.i.d. standard normal points in k-dimensional space?
• RQ2 How do the mean, variance, skewness, and kurtosis of the distance distribution change as the number of dimensions k increases?
• RQ3 To what extent does the inclusion of irrelevant, normally distributed features distort distance-based similarity measures in high-dimensional data?
• RQ4 Can the derived distribution be used to distinguish between meaningful nearest neighbors and those arising purely from random noise in high-dimensional datasets?
• RQ5 What is the theoretical behavior of distance distributions when k → ∞, and how does this affect the reliability of proximity-based learning algorithms?

Key findings

• The n-th raw moment of the distance R between two k-variate standard normal points is given by E[R^n] = 2^{1-k} Γ((k+n)/2) / Γ(k/2), providing a closed-form expression for all n ≥ 0.
• The second central moment (variance) of R is μ₂ = 2k − 4Γ((k+1)/2)² / Γ(k/2)², showing increasing variance with dimensionality.
• The third central moment μ₃ is expressed as [16Γ((k+1)/2)³ + 4(1−2k)Γ(k/2)²Γ((k+1)/2)] / Γ(k/2)³, indicating non-zero skewness that evolves with k.
• The fourth central moment μ₄ is derived as 4k(k+2) + [π·4^{3−k}(k−2)Γ(k)² − 48Γ((k+1)/2)⁴] / Γ(k/2)⁴, enabling calculation of kurtosis.
• Skewness γ₁ and kurtosis β₂ are derived in closed form using the central moments, revealing that the distribution becomes increasingly symmetric and leptokurtic as k increases.
• The results demonstrate that in high dimensions, distances between random Gaussian points become highly concentrated around a mean value, undermining the discriminative power of nearest-neighbor methods.