[Paper Review] Approximate Inference in Continuous Determinantal Point Processes

Raja Hafiz Affandi, Emily B. Fox+1 more|arXiv (Cornell University)|Nov 12, 2013

Point processes and geometric inequalities22 references15 citations

TL;DR

This paper proposes efficient approximate sampling methods for continuous determinantal point processes (DPPs), leveraging low-rank kernel approximations via Nyström and random Fourier features, along with a Gibbs sampler for k-DPPs. The approach enables tractable inference in high-dimensional continuous spaces, demonstrating utility in repulsive mixture modeling and diverse human pose synthesis with superior coverage compared to i.i.d. sampling.

ABSTRACT

Determinantal point processes (DPPs) are random point processes well-suited for modeling repulsion. In machine learning, the focus of DPP-based models has been on diverse subset selection from a discrete and finite base set. This discrete setting admits an efficient sampling algorithm based on the eigendecomposition of the defining kernel matrix. Recently, there has been growing interest in using DPPs defined on continuous spaces. While the discrete-DPP sampler extends formally to the continuous case, computationally, the steps required are not tractable in general. In this paper, we present two efficient DPP sampling schemes that apply to a wide range of kernel functions: one based on low rank approximations via Nystrom and random Fourier feature techniques and another based on Gibbs sampling. We demonstrate the utility of continuous DPPs in repulsive mixture modeling and synthesizing human poses spanning activity spaces.

Motivation & Objective

• To address the lack of tractable sampling algorithms for continuous DPPs, which hinders their use in modeling repulsion in continuous spaces.
• To extend the computational efficiency of discrete DPPs—based on eigendecomposition—to continuous domains through kernel approximation techniques.
• To enable practical application of continuous DPPs in generative modeling and diverse subset selection for high-dimensional data.
• To develop both low-rank approximation-based and Gibbs sampling-based inference schemes applicable to a wide range of kernel functions.
• To empirically validate the methods on real-world problems such as human pose synthesis and repulsive mixture modeling.

Proposed method

• Utilizes a dual representation of the DPP to enable sampling via low-rank kernel approximations, avoiding full eigendecomposition of the kernel operator.
• Employs Nyström and random Fourier feature (RFF) methods to approximate the kernel function, enabling efficient computation of the DPP likelihood and sampling.
• Derives a Gibbs sampler for k-DPPs using the Schur complement of the kernel matrix to compute conditional distributions over remaining points.
• Applies the low-rank approximation to the kernel in a dual form, allowing sampling from the DPP without explicit eigen-decomposition of the full kernel operator.
• Uses the resulting approximate DPP as a proposal distribution, with potential correction via Metropolis-Hastings to reduce approximation error.
• Employs a continuous-space DPP formulation where the probability density is proportional to the determinant of the kernel matrix over selected points.

Experimental results

Research questions

• RQ1 Can low-rank kernel approximations such as Nyström and random Fourier features be effectively used to enable efficient sampling from continuous DPPs?
• RQ2 How can the Gibbs sampling framework be adapted for k-DPPs in continuous spaces using the Schur complement of the kernel?
• RQ3 To what extent do approximate DPP samplers outperform i.i.d. sampling in covering high-dimensional data manifolds, such as in human motion synthesis?
• RQ4 Can continuous DPPs be effectively applied to repulsive mixture modeling, and how do they compare to standard non-repulsive mixture models?
• RQ5 What are the computational scaling properties of the proposed methods with respect to dimensionality d?

Key findings

• The low-rank approximation method using RFF and Nyström achieves high-quality DPP sampling with linear scaling in dimension d, enabling use in high-dimensional spaces such as 62D human motion data.
• In human pose synthesis, DPP-sampled poses achieved 90% data coverage at ε=40, outperforming i.i.d. sampling which required ε=253 to cover all data.
• The DPP-based method covered more rare poses than i.i.d. sampling, demonstrating improved diversity and robustness in high-dimensional activity spaces.
• For repulsive mixture modeling, the DPP-based model achieved lower classification error (0.15 vs. 0.23) on the two-class Iris dataset compared to a standard non-repulsive mixture model.
• The Gibbs sampler for k-DPPs provided exact conditional sampling by leveraging the Schur complement, enabling precise inference in structured continuous spaces.
• Empirical analysis confirmed that the low-rank approximations preserved the repulsive properties of the DPP, with minimal degradation in sampling quality across tested kernels.