[Paper Review] Wave function representation of probability distributions

Madeleine B. Thompson|arXiv (Cornell University)|Dec 21, 2017

Probabilistic and Robust Engineering Design10 references3 citations

TL;DR

This paper introduces a wave function representation of probability distributions using orthogonal Hermite basis expansions to model the square root of a density, enabling flexible, low-dimensional, and overfit-resistant modeling of continuous univariate distributions. The method enables efficient maximum likelihood estimation of coefficients via stereographic projection, supports exact moment computation, and outperforms kernel density estimation in tail behavior and tuning-free fitting.

ABSTRACT

Orthogonal decomposition of the square root of a probability density function in the Hermite basis is a useful low-dimensional parameterization of continuous probability distributions over the reals. This representation is formally similar to the representation of quantum mechanical states as wave functions, whose squared modulus is a probability density.

Motivation & Objective

• To develop a low-dimensional, flexible, and robust parameterization of continuous univariate probability distributions for use in data analysis systems.
• To address the lack of a general, natural, and computationally efficient family of distributions that can model skewed, multimodal, and heavy-tailed data without tuning parameters.
• To enable accurate modeling of regression residuals in a production data analysis platform (EDP) with minimal customization.
• To provide a parameterization that supports exact computation of moments and efficient sampling, avoiding the drawbacks of kernel density estimation.
• To explore the potential of extending this representation to conditional models and other domains using different orthogonal bases.

Proposed method

• Represent the square root of a probability density function as a linear combination of orthonormal Hermite functions, enabling a wave function-like parameterization.
• Use maximum likelihood estimation (MLE) of coefficients $ w_k $ via stereographic projection to transform the unit sphere constraint into an unconstrained space for optimization.
• Apply L-BFGS optimization to fit coefficients from i.i.d. samples, with the log-likelihood expressed in terms of the inverse stereographic projection of $ \gamma $.
• Compute moments exactly using Gauss–Hermite quadrature in $ O(K) $ operations, leveraging the polynomial-exponential form of the density estimate.
• Sample from the estimated density using a univariate slice sampler, with plans to switch to Adaptive Rejection Metropolis Sampling (ARMS) for improved efficiency.
• Normalize input data to have mean zero and variance 1/2 to stabilize the representation and enable fixed initial slice width in sampling.

Experimental results

Research questions

• RQ1 Can a Hermite-based wave function representation provide a flexible, low-dimensional, and overfit-resistant parameterization of arbitrary continuous univariate probability distributions?
• RQ2 How well can this method model skewed, multimodal, and heavy-tailed distributions such as Student’s t, uniform, and beta distributions?
• RQ3 Does the wave function representation outperform kernel density estimation in terms of tail behavior, computational efficiency, and lack of tuning parameters?
• RQ4 Can the MLE of coefficients be efficiently computed and stabilized using stereographic projection to handle the unit sphere constraint?
• RQ5 To what extent can this method be extended to conditional models where coefficients depend on predictors for heteroskedastic regression?

Key findings

• The wave function representation accurately models a wide range of distributions, including Student’s t with 4 degrees of freedom, bimodal mixtures, and skewed Beta(3,5), with minimal overfitting.
• The method achieves good fits even for challenging distributions like the uniform, though Gibbs phenomena appear at discontinuities, and floating-point errors limit the maximum degree to around 20.
• The MLE of coefficients converges reliably even with small sample sizes (e.g., n=2), and L-BFGS optimization performs well in practice.
• Moments can be computed exactly using Gauss–Hermite quadrature in $ O(K) $ operations, offering high accuracy and efficiency.
• The estimated density $ \hat{f}(x) $ has well-behaved tails and supports efficient sampling, with EDP using a univariate slice sampler and plans to adopt ARMS for speed.
• The representation is robust to model complexity: increasing the degree $ K $ improves fit for multimodal and discontinuous densities, though convergence is limited by numerical precision.