[Paper Review] Learning Determinantal Point Processes

Alex Kulesza, Ben Taskar|Scholarly Commons (University of Pennsylvania)|Feb 14, 2012

Stochastic processes and statistical mechanics16 references108 citations

TL;DR

This paper proposes a feature-based parameterization of conditional determinantal point processes (DPPs) that enables convex optimization for end-to-end learning from labeled data. By modeling diverse, non-redundant subset selection through a tractable, discriminative framework, the method achieves state-of-the-art performance in extractive multi-document summarization on the DUC 2003/04 benchmark, effectively balancing relevance and diversity.

ABSTRACT

Determinantal point processes (DPPs), which arise in random matrix theory and quantum physics, are natural models for subset selection problems where diversity is preferred. Among many remarkable properties, DPPs offer tractable algorithms for exact inference, including computing marginal probabilities and sampling; however, an important open question has been how to learn a DPP from labeled training data. In this paper we propose a natural feature-based parameterization of conditional DPPs, and show how it leads to a convex and efficient learning formulation. We analyze the relationship between our model and binary Markov random fields with repulsive potentials, which are qualitatively similar but computationally intractable. Finally, we apply our approach to the task of extractive summarization, where the goal is to choose a small subset of sentences conveying the most important information from a set of documents. In this task there is a fundamental tradeoff between sentences that are highly relevant to the collection as a whole, and sentences that are diverse and not repetitive. Our parameterization allows us to naturally balance these two characteristics. We evaluate our system on data from the DUC 2003/04 multi-document summarization task, achieving state-of-the-art results.

Motivation & Objective

• To address the open problem of learning DPPs from labeled training data.
• To develop a feature-based parameterization that enables efficient and convex optimization for conditional DPPs.
• To balance relevance and diversity in extractive summarization by modeling both through a unified probabilistic framework.
• To demonstrate the effectiveness of the proposed method on real-world multi-document summarization tasks.

Proposed method

• The authors introduce a conditional DPP parameterization using a feature function that maps input configurations to a kernel matrix, enabling discriminative learning.
• They formulate the learning objective as a convex optimization problem over the kernel parameters, ensuring global convergence.
• The method uses a log-determinant function to model the DPP likelihood, which is tractable and differentiable for efficient gradient-based optimization.
• The framework is applied to extractive summarization by modeling sentence selection as a diverse subset selection problem.
• The model learns to balance high relevance to the document collection and low redundancy among selected sentences through joint feature representation.

Experimental results

Research questions

• RQ1 Can a discriminative, feature-based parameterization of DPPs be formulated to allow convex learning from labeled data?
• RQ2 How does the proposed DPP model compare to binary Markov random fields with repulsive potentials in terms of computational tractability and performance?
• RQ3 To what extent can the DPP framework balance relevance and diversity in extractive summarization tasks?
• RQ4 Does the proposed learning method achieve state-of-the-art results on standard multi-document summarization benchmarks?

Key findings

• The proposed feature-based DPP parameterization enables convex optimization, making end-to-end learning feasible and efficient.
• The method achieves state-of-the-art performance on the DUC 2003/04 multi-document summarization task, outperforming prior approaches.
• The model effectively balances sentence relevance and diversity, reducing redundancy while maintaining high information coverage.
• The framework is computationally tractable and provides exact inference for marginal probabilities and sampling, unlike intractable alternatives such as repulsive MRFs.