[Paper Review] Polarized User and Topic Tracking in Twitter

Mauro Coletto, Claudio Lucchese+2 more|arXiv (Cornell University)|Oct 27, 2016

Complex Network Analysis Techniques9 references10 citations

TL;DR

This paper proposes PTR (Polarization TRacker), an iterative algorithm that identifies polarized users and discovers discriminating hashtags in Twitter streams by leveraging initial seed hashtags. The method achieves up to 71% higher F-measure than k-means baseline after two iterations, with stable performance and high recall (85.3% on IT13), enabling effective tracking of polarized communities and topics during political events.

ABSTRACT

Digital traces of conversations in micro-blogging platforms and OSNs provide information about user opinion with a high degree of resolution. These information sources can be exploited to under- stand and monitor collective behaviors. In this work, we focus on polarization classes, i.e., those topics that require the user to side exclusively with one position. The proposed method provides an iterative classification of users and keywords: first, polarized users are identified, then polarized keywords are discovered by monitoring the activities of previously classified users. This method thus allows tracking users and topics over time. We report several experiments conducted on two Twitter datasets during political election time-frames. We measure the user classification accuracy on a golden set of users, and analyze the relevance of the extracted keywords for the ongoing political discussion.

Motivation & Objective

• To identify polarized users in social media streams where users must align with one of a few predefined positions.
• To discover the most relevant, discriminating hashtags that characterize each polarization class.
• To enable continuous tracking of user communities and evolving discussion topics over time in dynamic social media environments.
• To evaluate the method on real-world Twitter datasets during political election periods with a golden set of labeled users.

Proposed method

• The PTR algorithm starts with a single seed hashtag per polarization class and iteratively identifies users who frequently use these hashtags.
• It extracts new, more discriminating hashtags from the tweets of users classified as polarized in each iteration.
• User classification is updated in each iteration based on the evolving set of discovered hashtags, improving precision and recall.
• The temporal variant, TPTR, processes data day-by-day, reclassifying users and updating hashtags to adapt to concept drift and emerging trends.
• The method operates as a semi-supervised clustering approach with minimal initial supervision—only the number of classes and one keyword per class are required.
• Evaluation uses F-measure, precision, recall, and coverage (γ) to assess classification quality against a golden set of labeled users.

Experimental results

Research questions

• RQ1 How can polarized users be effectively identified in Twitter streams when only minimal initial knowledge (e.g., one keyword per class) is available?
• RQ2 What is the impact of iterative hashtag discovery on the accuracy and coverage of user classification in polarized communities?
• RQ3 How well can the method track evolving user communities and discussion topics over time in dynamic social media environments?
• RQ4 To what extent does the iterative refinement of hashtags improve the detection of relevant topics within polarized user clusters?

Key findings

• The PTR algorithm achieved an F-measure of 0.588 on the IT13 dataset and 0.662 on the EU14 dataset after four iterations, showing significant improvement over the k-means baseline.
• After two iterations, the algorithm reached 84.5% coverage (γ) on IT13 and 83.0% on EU14, with F-measure improvements of +71% and +7% respectively over k-means.
• The second iteration yielded the largest performance gain, with recall increasing significantly due to the discovery of new, more discriminating hashtags.
• The TPTR variant showed a steady increase in F-measure from 0.177 on day 1 to 0.391 on day 9 for IT13, and from 0.155 to 0.635 for EU14, reflecting improved classification over time.
• The method successfully extracted meaningful keywords such as prominent politicians, party names, and political slogans, confirming the relevance of discovered topics.
• The algorithm stabilized quickly, with minimal performance gains after the second iteration, indicating that two iterations are sufficient for high-quality classification.