[Paper Review] SybilBlind: Detecting Fake Users in Online Social Networks without Manual Labels

Binghui Wang, Le Zhang+1 more|arXiv (Cornell University)|Jun 13, 2018

Spam and Phishing Detection2 references8 citations

TL;DR

SybilBlind is a novel, structure-based Sybil detection framework that detects fake users in online social networks without requiring manually labeled training data. It uses multiple random sampling trials with noisy labels and a homophily-entropy aggregator to identify low-noise trials, achieving an AUC of 0.98 on a real Twitter dataset, outperforming methods needing manual labeling by requiring only 2.8% of nodes labeled to match its performance.

ABSTRACT

Detecting fake users (also called Sybils) in online social networks is a basic security research problem. State-of-the-art approaches rely on a large amount of manually labeled users as a training set. These approaches suffer from three key limitations: 1) it is time-consuming and costly to manually label a large training set, 2) they cannot detect new Sybils in a timely fashion, and 3) they are vulnerable to Sybil attacks that leverage information of the training set. In this work, we propose SybilBlind, a structure-based Sybil detection framework that does not rely on a manually labeled training set. SybilBlind works under the same threat model as state-of-the-art structure-based methods. We demonstrate the effectiveness of SybilBlind using 1) a social network with synthetic Sybils and 2) two Twitter datasets with real Sybils. For instance, SybilBlind achieves an AUC of 0.98 on a Twitter dataset.

Motivation & Objective

• To address the limitations of state-of-the-art Sybil detection methods that rely on costly, time-consuming, and privacy-invasive manual labeling of training data.
• To develop a framework that detects Sybils without access to any manually labeled users, while remaining robust to adversarial training set inference.
• To enable timely detection of new Sybil attacks that emerge after existing ones are mitigated.
• To design a robust aggregation mechanism that identifies high-quality sampling trials despite label noise in the training sets.

Proposed method

• Randomly sample a subset of users from the social network and assign them random labels (benign or Sybil), creating a noisy training set without actual manual labeling.
• Use a state-of-the-art Sybil detection method, such as SybilSCAR, on each sampled noisy training set to generate a Sybil probability score for every user in that trial.
• Design a novel aggregator that uses two metrics—homophily and one-side entropy—to identify sampling trials with low label noise where Sybil detection is accurate.
• Aggregate Sybil probabilities only from high-quality trials identified by the homophily-entropy metrics to produce a final, reliable detection score for each user.
• Repeat the sampling and detection process across multiple independent trials to improve robustness and accuracy.
• Apply the framework to real-world datasets, including Twitter with real Sybils, and synthetic networks to validate performance.

Experimental results

Research questions

• RQ1 Can Sybil detection be achieved without any manually labeled training data while maintaining high accuracy?
• RQ2 How can a reliable detection model be constructed when training labels are randomly assigned and inherently noisy?
• RQ3 What aggregation strategy can effectively identify high-quality sampling trials with low label noise in the absence of ground truth?
• RQ4 How does the performance of SybilBlind compare to supervised methods that require manual labeling, especially in terms of labeling cost and detection speed?
• RQ5 What is the impact of varying Sybil fractions and network structure on the detection accuracy of the proposed framework?

Key findings

• SybilBlind achieves an AUC of 0.98 on a real Twitter dataset with 8,000 users and 68,000 edges, demonstrating high detection accuracy without manual labeling.
• On a large Twitter dataset of 42 million users and 1.2 billion edges, SybilBlind maintains strong performance, confirming scalability.
• The homophily-entropy aggregator significantly outperforms average, min, and max aggregators, with the average aggregator performing close to random guessing.
• SybilBlind can detect Sybils accurately when their fraction is below 50%, which holds in most real-world scenarios where Sybils are outnumbered by benign users.
• SybilSCAR requires labeling approximately 25% of nodes in the small Twitter dataset and 2.8% in the large Twitter dataset to match SybilBlind’s AUC, highlighting the cost advantage of the proposed method.
• The framework’s performance stabilizes with sufficient sampling size and number of trials, with AUC increasing and then plateauing as these parameters grow.