[Paper Review] The nonparametric Fisher geometry and the chi-square process density prior

Andrew J. Holbrook, Shiwei Lan+2 more|arXiv (Cornell University)|Jul 11, 2017

Bayesian Methods and Mixture Models34 references3 citations

TL;DR

This paper introduces a novel Bayesian nonparametric density estimation model using the nonparametric Fisher geometry, which is shown to be equivalent to the infinite-dimensional sphere of square-root densities. By modeling the square-root density with a restricted Gaussian process prior and leveraging Karhunen-Loève expansions, the method enables fast inference via spherical Hamiltonian Monte Carlo, yielding a χ²-process density prior that ensures computational efficiency and posterior flexibility while formalizing the link between spherical HMC and Riemannian HMC in the infinite-dimensional limit.

ABSTRACT

It is well known that the Fisher information induces a Riemannian geometry on parametric families of probability density functions. Following recent work, we consider the nonparametric generalization of the Fisher geometry. The resulting nonparametric Fisher geometry is shown to be equivalent to a familiar, albeit infinite-dimensional, geometric object---the sphere. By shifting focus away from density functions and toward \emph{square-root} density functions, one may calculate theoretical quantities of interest with ease. More importantly, the sphere of square-root densities is much more computationally tractable. This insight leads to a novel Bayesian nonparametric density estimation model. We construct the $χ^2$-process density prior by modeling the square-root density with a restricted Gaussian process prior. Inference over square-root densities is fast, and the model retains the flexibility characteristic of Bayesian nonparametric models. Finally, we formalize the relationship between spherical HMC in the infinite-dimensional limit and standard Riemannian HMC.

Motivation & Objective

• To extend Fisher information geometry beyond parametric models to nonparametric density families.
• To establish the equivalence between nonparametric Fisher geometry and the infinite-dimensional L² sphere of square-root densities.
• To develop a computationally efficient Bayesian nonparametric density estimation model with full posterior inference capabilities.
• To formalize the connection between spherical HMC in finite dimensions and Riemannian HMC in the infinite-dimensional limit.
• To demonstrate the practical utility of the χ²-process prior through posterior sampling and applications in density estimation and point processes.

Proposed method

• The paper models the square-root density using a Gaussian process prior restricted to the infinite-dimensional sphere via a Dirac measure.
• It employs the Karhunen-Loève (K-L) expansion of the GP prior to represent the square-root density in terms of orthonormal eigenfunctions.
• The K-L expansion is truncated to reduce inference to a finite-dimensional sphere of coefficients, enabling efficient computation.
• Spherical Hamiltonian Monte Carlo (HMC) is used for posterior sampling over the finite-dimensional sphere of K-L coefficients.
• The square of the GP prior yields a χ²-process density prior, which is used for Bayesian density estimation.
• The method formalizes the equivalence between spherical HMC and Riemannian HMC in the infinite-dimensional limit through geodesic dynamics.

Experimental results

Research questions

• RQ1 How does the nonparametric Fisher geometry relate to the infinite-dimensional sphere of square-root densities?
• RQ2 Can a Gaussian process prior on square-root densities be restricted to the sphere via K-L expansion to enable efficient inference?
• RQ3 What is the relationship between spherical HMC on the finite-dimensional sphere and Riemannian HMC in the infinite-dimensional limit?
• RQ4 How does the χ²-process density prior perform in Bayesian density estimation compared to existing methods?
• RQ5 Can the χ²-process prior be effectively used in hierarchical models such as the Cox process for point process intensity estimation?

Key findings

• The nonparametric Fisher geometry on the space of probability densities is mathematically equivalent to the infinite-dimensional L² sphere of square-root densities.
• The use of square-root densities transforms the geometry into a sphere, enabling computationally tractable inference via spherical HMC.
• The K-L expansion of the GP prior allows exact restriction to the sphere and enables linear scaling of computational complexity with data size.
• The χ²-process density prior, derived from the squared GP prior, enables fast posterior sampling and produces plausible density realizations with minimal assumptions.
• Spherical HMC on the finite-dimensional K-L coefficient sphere corresponds to Riemannian HMC in the infinite-dimensional limit, formalizing a key theoretical link.
• The model achieves efficient posterior computation and is applicable to complex models such as the Cox process, where the intensity function is modeled as a product of a scale parameter and a χ²-process density.