[Paper Review] Dynamic Ranking with the BTL Model: A Nearest Neighbor based Rank Centrality Method

Eglantine Karlé, Hemant Tyagi|arXiv (Cornell University)|Sep 28, 2021

Topological and Geometric Data Analysis25 references4 citations

TL;DR

This paper proposes a dynamic ranking method based on the Bradley-Terry-Luce (BTL) model using a nearest-neighbor weighted Rank Centrality approach to estimate time-varying item strengths from evolving pairwise comparison graphs. It establishes non-asymptotic $ε_2$ and $ε_\infty$ error bounds under Erdös-Rényi graph assumptions, proving consistency in estimating dynamic item rankings over time.

ABSTRACT

Many applications such as recommendation systems or sports tournaments involve pairwise comparisons within a collection of $n$ items, the goal being to aggregate the binary outcomes of the comparisons in order to recover the latent strength and/or global ranking of the items. In recent years, this problem has received significant interest from a theoretical perspective with a number of methods being proposed, along with associated statistical guarantees under the assumption of a suitable generative model. While these results typically collect the pairwise comparisons as one comparison graph $G$, however in many applications - such as the outcomes of soccer matches during a tournament - the nature of pairwise outcomes can evolve with time. Theoretical results for such a dynamic setting are relatively limited compared to the aforementioned static setting. We study in this paper an extension of the classic BTL (Bradley-Terry-Luce) model for the static setting to our dynamic setup under the assumption that the probabilities of the pairwise outcomes evolve smoothly over the time domain $[0,1]$. Given a sequence of comparison graphs $(G_{t'})_{t' \\in \\mathcal{T}}$ on a regular grid $\\mathcal{T} \\subset [0,1]$, we aim at recovering the latent strengths of the items $w_t^* \\in \\mathbb{R}^n$ at any time $t \\in [0,1]$. To this end, we adapt the Rank Centrality method - a popular spectral approach for ranking in the static case - by locally averaging the available data on a suitable neighborhood of $t$. When $(G_{t'})_{t' \\in \\mathcal{T}}$ is a sequence of Erd\\"os-Renyi graphs, we provide non-asymptotic $\\ell_2$ and $\\ell_{\\infty}$ error bounds for estimating $w_t^*$ which in particular establishes the consistency of this method in terms of $n$, and the grid size $\\lvert\\mathcal{T}\ vert$. We also complement our theoretical analysis with experiments on real and synthetic data.

Motivation & Objective

• To address the lack of theoretical guarantees in dynamic ranking scenarios where pairwise comparison graphs evolve over time.
• To extend the static Rank Centrality method to a dynamic setting with time-varying item strengths under the BTL model.
• To provide non-asymptotic $ε_2$ and $ε_\infty$ error bounds for estimating time-dependent item strengths $w_t^*$.
• To ensure uniform consistency over the time interval $[0,1]$ using local averaging and union bound arguments.
• To validate the method empirically on both synthetic and real-world data.

Proposed method

• Adapts the static Rank Centrality spectral method by locally averaging comparison data over a neighborhood of time $t$ using a nearest-neighbor kernel.
• Constructs an estimated transition matrix $\widehat{P}(t)$ at each time $t$ based on comparison graphs $G_{t'}$ within a time window around $t$.
• Uses the top eigenvector of $\widehat{P}(t)$ to estimate the normalized ranking vector $\widehat{\pi}(t)$ at time $t$.
• Applies a union bound over $O(T)$ time points to establish uniform error bounds across the entire time interval $[0,1]$.
• Assumes smooth evolution of item strengths over time and models the comparison graphs as realizations of Erdös-Rényi random graphs.
• Employs a Lipschitz continuity assumption on the true ranking vector $\pi^*(t)$ to control temporal variation in the estimation error.

Experimental results

Research questions

• RQ1 Can a spectral ranking method be adapted to estimate time-varying item strengths in a dynamic BTL model with smooth temporal evolution?
• RQ2 What non-asymptotic error bounds can be derived for the estimated ranking vector in the dynamic setting?
• RQ3 How does local averaging over time neighborhoods affect the consistency and accuracy of dynamic ranking estimation?
• RQ4 Under what conditions does the nearest-neighbor Rank Centrality method achieve uniform consistency over the time interval $[0,1]$?
• RQ5 How do the error bounds scale with the number of items $n$, the grid size $|\mathcal{T}|$, and the number of comparisons per pair?

Key findings

• The proposed nearest-neighbor Rank Centrality method achieves uniform $\ell_2$ error bounds of order $O\left(\frac{1}{\sqrt{n p L}}\right)$ with high probability, where $p$ is the edge probability and $L$ is the number of comparisons per pair.
• The method achieves uniform $\ell_\infty$ error bounds of order $O\left(\sqrt{\frac{\log n}{n p L}}\right)$, which enables exact recovery of the top-$K$ ranking under suitable conditions.
• Theoretical analysis establishes consistency of the method in terms of $n$ and the grid size $|\mathcal{T}|$, under the assumption that comparison graphs are Erdös-Rényi random graphs.
• The error bounds hold uniformly over $t \in [0,1]$ with probability at least $1 - O(T n^{-c})$, where $c$ is a large constant (e.g., 9 or 10).
• The method is shown to be robust to temporal variation through the use of local averaging and Lipschitz continuity of the true ranking vector $\pi^*(t)$.
• Empirical results on synthetic and real data validate the method’s performance and support the theoretical findings.