[Paper Review] Iterative Filtering for a Dynamical Reputation System

Cristobald de Kerchove, Paul Van Dooren|ArXiv.org|Nov 26, 2007

Spam and Phishing Detection11 references19 citations

TL;DR

This paper proposes an iterative filtering algorithm for a dynamical reputation system that jointly computes reputations for items and reliability weights for raters using a nonlinear, sparsity-aware model. The method superlinearly converges to a unique solution, robustly downweighting outliers like spammers and random raters without discarding data, with linear complexity per iteration and strong empirical resilience to malicious inputs.

ABSTRACT

The paper introduces a novel iterative method that assigns a reputation to n + m items: n raters and m objects. Each rater evaluates a subset of objects leading to a n x m rating matrix with a certain sparsity pattern. From this rating matrix we give a nonlinear formula to define the reputation of raters and objects. We also provide an iterative algorithm that superlinearly converges to the unique vector of reputations and this for any rating matrix. In contrast to classical outliers detection, no evaluation is discarded in this method but each one is taken into account with different weights for the reputation of the objects. The complexity of one iteration step is linear in the number of evaluations, making our algorithm efficient for large data set. Experiments show good robustness of the reputation of the objects against cheaters and spammers and good detection properties of cheaters and spammers.

Motivation & Objective

• To address the challenge of assigning reliable reputations to items in large-scale rating systems where raters may be unreliable or malicious.
• To develop a method that simultaneously estimates item reputations and rater reliability without discarding any evaluations, unlike traditional outlier detection.
• To ensure convergence to a unique solution under arbitrary sparsity patterns in the rating matrix, even with adversarial raters.
• To provide a scalable, efficient algorithm with linear complexity per iteration suitable for large datasets.
• To improve robustness in collaborative filtering and trust systems by downweighting inconsistent raters through belief divergence-based trust scores.

Proposed method

• The method models reputations using a nonlinear formula based on belief divergence between a rater's evaluations and the current item reputations.
• It introduces a trust matrix T where each entry T_ij = c_j - d_i, with d_i being the belief divergence of rater i, to weight evaluations according to rater reliability.
• The algorithm iteratively refines reputations and trust scores using a fixed-point iteration that superlinearly converges to a unique solution.
• The belief divergence d_i is computed as the L2 distance between a rater’s evaluations and the current item reputations, serving as a measure of inconsistency.
• The method uses a flexible parameter c_j to control sensitivity to divergence, allowing interpolation between uniform weighting and aggressive outlier detection.
• Convergence is guaranteed for any initial rating matrix and any sparsity pattern, with computational complexity linear in the number of evaluations.

Experimental results

Research questions

• RQ1 How can we jointly estimate item reputations and rater reliability in a way that is robust to malicious or inconsistent raters?
• RQ2 Can we design an iterative algorithm that superlinearly converges to a unique solution without discarding any data?
• RQ3 To what extent does the method preserve accurate item reputations when spammers or random raters are introduced?
• RQ4 How do multiple iterations improve the separation of honest raters from outliers compared to a single-step approach?
• RQ5 What is the impact of different trust functions (e.g., exponential or inverse) on the final reputation scores and convergence properties?

Key findings

• The iterative filtering algorithm superlinearly converges to a unique solution for any rating matrix, regardless of sparsity or initial conditions.
• After convergence, the reputations of items are significantly more robust to spam: the 1-norm difference between pre- and post-spam reputations is 267 when using the algorithm, compared to 638 when using simple averaging.
• One iteration provides poor discrimination between honest and malicious raters; convergence is essential for reliable outlier detection, as shown in Figure 6.
• Spammers and random raters are naturally downweighted due to high belief divergence, with their trust scores T_ij approaching zero.
• The method effectively separates honest raters from outliers across multiple iterations, with increasing clarity in the trust distribution as seen in the density plots of Figure 6.
• The approach is scalable, with each iteration requiring only linear time in the number of evaluations, making it suitable for large-scale systems.