[Paper Review] Private Rank Aggregation under Local Differential Privacy

Ziqi Yan, Gang Li+1 more|arXiv (Cornell University)|Aug 13, 2019

Privacy-Preserving Technologies in Data46 references4 citations

TL;DR

This paper proposes LDP-KwikSort, a locally differentially private rank aggregation protocol that protects agents' pairwise preference data by applying randomized response or Laplace noise to individual responses before sending them to an untrusted curator. The method achieves strong privacy with acceptable utility, outperforming Laplace-based variants and approaching central-model DP performance.

ABSTRACT

As a method for answer aggregation in crowdsourced data management, rank aggregation aims to combine different agents' answers or preferences over the given alternatives into an aggregate ranking which agrees the most with the preferences. However, since the aggregation procedure relies on a data curator, the privacy within the agents' preference data could be compromised when the curator is untrusted. Existing works that guarantee differential privacy in rank aggregation all assume that the data curator is trusted. In this paper, we formulate and address the problem of locally differentially private rank aggregation, in which the agents have no trust in the data curator. By leveraging the approximate rank aggregation algorithm KwikSort, the Randomized Response mechanism, and the Laplace mechanism, we propose an effective and efficient protocol LDP-KwikSort. Theoretical and empirical results show that the solution LDP-KwikSort:RR can achieve the acceptable trade-off between the utility of aggregate ranking and the privacy protection of agents' pairwise preferences.

Motivation & Objective

• Address the lack of privacy protection in rank aggregation when the data curator is untrusted, a critical gap in existing differentially private approaches.
• Formulate and solve the locally differentially private rank aggregation (LDP-RA) problem, where agents cannot trust the curator with raw preference data.
• Design an efficient protocol that protects pairwise comparisons rather than full rankings, reducing privacy leakage and improving utility.
• Evaluate trade-offs between privacy budget, number of queries, and utility in the local differential privacy model.
• Demonstrate that the proposed protocol maintains aggregate ranking quality close to non-private baselines under strong privacy guarantees.

Proposed method

• Leverage the KwikSort algorithm, which operates on pairwise preferences rather than full rankings, to reduce the sensitivity of the aggregation process.
• Apply the Randomized Response (RR) mechanism or Laplace mechanism as local randomizers to perturb agents’ pairwise comparison answers before transmission.
• Use the curator’s post-processing to estimate the aggregate pairwise comparison frequencies from noisy reports.
• Construct the final aggregate ranking using KwikSort on the estimated pairwise frequencies.
• Theoretical analysis derives an upper bound on the estimation error of the aggregate pairwise profile, guiding optimal query count $K$ selection.
• Optimize the number of queries $K$ to balance utility and privacy, showing that utility peaks around $K = \sqrt{m}$ for $m$ alternatives.

Experimental results

Research questions

• RQ1 How can we design a rank aggregation protocol that ensures local differential privacy when the data curator is untrusted?
• RQ2 What is the optimal number of pairwise queries $K$ that maximizes utility while preserving privacy in the local model?
• RQ3 How do the Randomized Response and Laplace mechanisms compare in terms of utility and error for LDP rank aggregation?
• RQ4 To what extent can the proposed protocol preserve the accuracy of the aggregate ranking under varying privacy budgets and dataset characteristics?
• RQ5 How does the dispersion parameter $\theta$ of the Mallows model affect the performance of the LDP-KwikSort protocol?

Key findings

• LDP-KwikSort:RR consistently outperforms LDP-KwikSort:Lap in terms of utility, achieving lower average Kendall tau distance and error rates.
• When $n=2500$, $m=15$, and $\theta=0.5$, the gap in error rate between LDP-KwikSort:RR and LDP-KwikSort:Lap is 2.4%, increasing to 32.5% at $m=45$.
• With $n=5000$, $m=45$, and increasing $\theta$ from 0.25 to 0.75, the performance gap between LDP-KwikSort:RR and LDP-KwikSort:Lap grows from 13.5% to 46.5% in average Kendall tau distance.
• For $m \leq 15$, $\epsilon = 1.0$ provides acceptable utility; for larger $m$, $\epsilon = 3.0$ is recommended to maintain performance.
• Time cost increases with the number of alternatives, and LDP-KwikSort:RR becomes slower than LDP-KwikSort:Lap when $m > 15$, due to higher query volume.
• The protocol achieves $\epsilon$-local differential privacy and maintains utility close to the central-model DP-KwikSort, especially under optimal $K$ settings.