[Paper Review] Bayesian inference from rank data

Øystein Sørensen, Valeria Vitelli+2 more|arXiv (Cornell University)|May 30, 2014

Game Theory and Voting Systems31 references2 citations

TL;DR

This paper develops computationally efficient Bayesian inference methods for Mallows rank models using any right-invariant metric, enabling flexible analysis of rank data including top-t rankings, pairwise comparisons, and time-varying rankings. The approach supports consensus estimation, clustering of assessors, and regression over time, with demonstrated performance on simulated, experimental, and benchmark data.

ABSTRACT

Modeling and analysis of rank data have received renewed interest in the era of big data, when recruited and volunteer assessors compare and rank objects to facilitate decision mak-ing in disparate areas, from politics to entertainment, from education to marketing. The Mallows rank model is among the most successful approaches, but its use has been limited to a particular form based on the Kendall distance, for which the normalizing constant has a simple analytic expression. In this paper, we develop computationally tractable methods for performing Bayesian inference in Mallows models with any right-invariant metric, thereby allowing for greatly extended flexibility. Our methods also allow to estimate consensus rank-ings for data in the form of top-t rankings, pairwise comparisons, and other linear orderings. In addition, clustering via mixtures allows to find substructure among assessors. Finally we construct a regression framework for ranks which vary over time. We illustrate and investigate our approach on simulated data, on performed experiments, and on benchmark data.

Motivation & Objective

• To extend Bayesian inference in Mallows models beyond the traditional Kendall distance to any right-invariant metric, enhancing modeling flexibility.
• To enable estimation of consensus rankings from incomplete or partial rank data, such as top-t rankings and pairwise comparisons.
• To support clustering of assessors by identifying subgroups with distinct ranking behaviors through mixture models.
• To develop a regression framework for modeling time-varying rankings, capturing dynamic preferences over time.
• To provide computationally tractable inference methods that scale to real-world rank data in big data settings.

Proposed method

• Leverages the mathematical structure of right-invariant metrics on permutation groups to generalize the Mallows model beyond the Kendall distance.
• Employs Markov Chain Monte Carlo (MCMC) methods with efficient proposal distributions tailored to the permutation space and right-invariant geometry.
• Uses a conjugate prior setup for the consensus ranking and scale parameter, enabling posterior computation via Gibbs sampling.
• Incorporates data augmentation techniques to handle top-t rankings and pairwise comparisons by modeling latent full rankings.
• Applies finite mixture models over assessors to detect subpopulations with distinct ranking behaviors.
• Develops a dynamic regression model where the consensus ranking evolves over time via a state-space formulation with time-dependent parameters.

Experimental results

Research questions

• RQ1 How can Bayesian inference in Mallows models be generalized to any right-invariant metric, not just the Kendall distance?
• RQ2 Can consensus rankings be accurately estimated from partial rank data such as top-t lists or pairwise comparisons?
• RQ3 To what extent can clustering of assessors reveal hidden substructures in rank data?
• RQ4 How can time-varying preferences in ranking data be modeled and inferred using a regression framework?
• RQ5 What is the computational and statistical performance of the proposed methods on real-world and simulated rank data?

Key findings

• The proposed Bayesian framework enables efficient inference for Mallows models with any right-invariant metric, overcoming the limitation of requiring a closed-form normalizing constant.
• The method achieves accurate consensus ranking estimation even from top-t rankings and pairwise comparisons, with demonstrated robustness in simulation studies.
• Clustering via mixture models successfully identifies distinct subpopulations of assessors with coherent ranking behaviors in both simulated and real data.
• The time-varying regression model captures evolving preferences over time, showing strong fit on benchmark datasets with temporal ranking trends.
• The computational methods scale effectively to large-scale rank data, with MCMC chains mixing well and posterior estimates converging reliably across experiments.