[Paper Review] Geometry of Log-Concave Density Estimation

Elina Robeva, Bernd Sturmfels+1 more|arXiv (Cornell University)|Apr 6, 2017

Topological and Geometric Data Analysis9 references4 citations

TL;DR

This paper establishes a geometric link between nonparametric statistics and geometric combinatorics by proving that every regular polyhedral subdivision of a point configuration in $\mathbb{R}^d$ arises as the optimal log-concave density estimator for some positive weight vector. The authors introduce the Samworth body—a continuous analogue of the secondary polytope—to characterize the space of weights yielding each subdivision, showing that coarser subdivisions are statistically more likely than finer ones under uniform sampling over weights.

ABSTRACT

Shape-constrained density estimation is an important topic in mathematical statistics. We focus on densities on $\mathbb{R}^d$ that are log-concave, and we study geometric properties of the maximum likelihood estimator (MLE) for weighted samples. Cule, Samworth, and Stewart showed that the logarithm of the optimal log-concave density is piecewise linear and supported on a regular subdivision of the samples. This defines a map from the space of weights to the set of regular subdivisions of the samples, i.e. the face poset of their secondary polytope. We prove that this map is surjective. In fact, every regular subdivision arises in the MLE for some set of weights with positive probability, but coarser subdivisions appear to be more likely to arise than finer ones. To quantify these results, we introduce a continuous version of the secondary polytope, whose dual we name the Samworth body. This article establishes a new link between geometric combinatorics and nonparametric statistics, and it suggests numerous open problems.

Motivation & Objective

• To establish a rigorous connection between maximum likelihood estimation of log-concave densities and the theory of regular polyhedral subdivisions.
• To prove that every regular subdivision of a point configuration in $\mathbb{R}^d$ arises as the optimal log-concave density estimator for some positive weight vector.
• To introduce and analyze the Samworth body—a continuous analogue of the secondary polytope—as a geometric object encoding the distribution of optimal subdivisions across weight space.
• To quantify the likelihood of different subdivisions arising under random weight sampling, showing that coarser subdivisions are more probable than finer ones.

Proposed method

• The authors generalize Cule, Samworth, and Stewart's result by allowing arbitrary positive weights in the maximum likelihood estimation problem for log-concave densities.
• They define the Samworth body $\mathcal{S}(X)$ as the convex subset of $\mathbb{R}^n$ corresponding to feasible weight vectors that yield a given regular subdivision.
• Using Lagrange multipliers and volume integrals, they derive an unconstrained optimization formulation of the log-likelihood problem via the dual of the Samworth body.
• They prove surjectivity of the map from weight vectors to regular subdivisions by constructing explicit weight vectors that realize any given subdivision.
• They introduce a continuous version of the secondary polytope and use it to model the probability distribution of subdivisions under uniform sampling over weight space.
• They analyze a critical configuration of $d+2$ points in $\mathbb{R}^d$ to prove that non-trivial subdivisions can arise even with symmetric weights, using explicit volume and exponential function identities.

Experimental results

Research questions

• RQ1 Can every regular polyhedral subdivision of a point configuration in $\mathbb{R}^d$ be realized as the optimal log-concave density estimator for some positive weight vector?
• RQ2 What is the geometric structure of the set of weight vectors that yield a given regular subdivision?
• RQ3 How does the probability of observing a particular subdivision vary with the complexity of the subdivision under uniform sampling over the weight simplex?
• RQ4 What is the minimal number of points $n$ in $\mathbb{R}^d$ such that a given number $c$ of cells can appear in the optimal subdivision with unit weights?
• RQ5 Which simplicial polytopes can be realized as the regular subdivision of a point configuration in $\mathbb{R}^d$ with uniform weights?

Key findings

• Every regular polyhedral subdivision of a point configuration $X \subset \mathbb{R}^d$ arises as the optimal log-concave density estimator for some positive weight vector $w \in \mathbb{R}^n_{>0}$, proving the surjectivity of the weight-to-subdivision map.
• The Samworth body $\mathcal{S}(X)$ is a convex body in $\mathbb{R}^n$ that parametrizes the set of weight vectors yielding a given regular subdivision, and its dual provides a continuous generalization of the secondary polytope.
• For $d \geq 2$, there exist configurations of $d+3$ points in $\mathbb{R}^d$ with unit weights that yield non-trivial (non-flat) subdivisions, demonstrating that complex structures can emerge even with symmetric weights.
• The probability of observing a coarser subdivision is higher than that of a finer one under uniform sampling over the weight simplex, indicating a statistical bias toward simpler structures.
• For a specific configuration of $d+2$ points in $\mathbb{R}^d$, the optimal subdivision is the triangulation omitting each of the first $d+1$ points in turn if and only if the weight of the last point exceeds $\frac{d+1}{d}$ times the weight of each of the others.
• The paper proves that the set of configurations for which the optimal subdivision is trivial is closed in the space of configurations, implying that non-trivial subdivisions are generic under small perturbations of the point set.