[Paper Review] On information projections between multivariate elliptical and location-scale families

Frank Nielsen|arXiv (Cornell University)|Jan 11, 2021

Advanced Statistical Methods and Models34 references4 citations

TL;DR

This paper establishes that the minimum f-divergence between a location-scale family and another location-scale family is invariant to the choice of location and scale parameters of the prescribed distribution. By leveraging the invariance under the multivariate location-scale group action, the authors reduce f-divergence calculations to canonical forms, enabling efficient Monte Carlo estimation and revealing a geometric interpretation in hyperbolic space via the Fisher-Rao metric.

ABSTRACT

We study information projections with respect to statistical $f$-divergences between any two location-scale families. We consider a multivariate generalization of the location-scale families which includes the elliptical and the spherical subfamilies. By using the action of the multivariate location-scale group, we show how to reduce the calculation of $f$-divergences between any two location-scale densities to canonical settings involving standard densities, and derive thereof fast Monte Carlo estimators of $f$-divergences with good properties. Finally, we prove that the minimum $f$-divergence between a prescribed density of a location-scale family and another location-scale family is independent of the prescribed location-scale parameter. We interpret geometrically this property.

Motivation & Objective

• To study information projections between multivariate elliptical and location-scale families using f-divergences.
• To derive invariance properties of f-divergences under the action of the multivariate location-scale group.
• To reduce f-divergence computation to canonical forms involving standard densities for efficient estimation.
• To prove that the minimum f-divergence projection is independent of the prescribed location-scale parameters.
• To provide a geometric interpretation of the invariance property using hyperbolic geometry and the Fisher-Rao metric.

Proposed method

• Utilizes the action of the multivariate location-scale group to transform arbitrary densities into canonical forms, simplifying f-divergence computation.
• Derives invariance of f-divergences under location-scale transformations (Theorem 1), enabling reduction to standard densities.
• Applies the Fisher-Rao metric to show that the statistical manifold of location-scale families has constant negative curvature, implying hyperbolic geometry.
• Constructs fast Monte Carlo estimators for f-divergences when closed-form computation is infeasible, leveraging the canonical reduction.
• Uses Möbius transformations and isomorphisms between SLℝ(2) and SUℂ(1,1) to map the Poincaré upper half-plane to the unit disk for geometric analysis.
• Establishes that the Fisher-Rao distance between two densities in a location-scale family corresponds to a scaled hyperbolic distance in the Poincaré model.

Experimental results

Research questions

• RQ1 Does the minimum f-divergence between a location-scale family and another location-scale family depend on the choice of parameters in the prescribed distribution?
• RQ2 Can f-divergences between arbitrary location-scale densities be reduced to canonical forms involving standard densities?
• RQ3 What is the geometric structure of the statistical manifold of location-scale families under the Fisher-Rao metric?
• RQ4 How can efficient Monte Carlo estimators be constructed for f-divergences between location-scale families?
• RQ5 What is the role of the multivariate location-scale group in simplifying information projection problems?

Key findings

• The minimum f-divergence between a prescribed location-scale family and another location-scale family is independent of the location and scale parameters of the prescribed distribution.
• f-divergences are invariant under the action of the multivariate location-scale group, allowing reduction to canonical settings with standard densities.
• The Fisher-Rao metric on a location-scale family induces a hyperbolic geometry with constant negative curvature, specifically κ = −1/b².
• For the multivariate normal family, the Fisher-Rao distance is proportional to the hyperbolic distance in the Poincaré upper half-plane with curvature κ = −1/2.
• For the Cauchy family, the Fisher-Rao distance corresponds to a scaled hyperbolic distance with curvature κ = −2.
• Efficient Monte Carlo estimators for f-divergences are constructed by exploiting the canonical reduction, ensuring good statistical properties.