[Paper Review] Event detection in Colombian security Twitter news using fine-grained latent topic analysis

Vladimir Vargas-Calderón, Nicolás Parra-A.+2 more|arXiv (Cornell University)|Nov 19, 2019

Social Media and Politics26 references4 citations

TL;DR

This paper proposes a fine-grained latent topic analysis method using FastText, K-means clustering, and UMAP to detect and segment security, violence, and crime-related events in Colombian Twitter news. It identifies 10 interpretable topics—such as murders, traffic accidents, and violence against women—enabling automatic classification and monitoring of underreported violent incidents.

ABSTRACT

Cultural and social dynamics are important concepts that must be understood in order to grasp what a community cares about. To that end, an excellent source of information on what occurs in a community is the news, especially in recent years, when mass media giants use social networks to communicate and interact with their audience. In this work, we use a method to discover latent topics in tweets from Colombian Twitter news accounts in order to identify the most prominent events in the country. We pay particular attention to security, violence and crime-related tweets because of the violent environment that surrounds Colombian society. The latent topic discovery method that we use builds vector representations of the tweets by using FastText and finds clusters of tweets through the K-means clustering algorithm. The number of clusters is found by measuring the $C_V$ coherence for a range of number of topics of the Latent Dirichlet Allocation (LDA) model. We finally use Uniform Manifold Approximation and Projection (UMAP) for dimensionality reduction to visualise the tweets vectors. Once the clusters related to security, violence and crime are identified, we proceed to apply the same method within each cluster to perform a fine-grained analysis in which specific events mentioned in the news are grouped together. Our method is able to discover event-specific sets of news, which is the baseline to perform an extensive analysis of how people engage in Twitter threads on the different types of news, with an emphasis on security, violence and crime-related tweets.

Motivation & Objective

• To identify and segment security, violence, and crime-related news tweets from Colombian Twitter media accounts.
• To enable fine-grained analysis of public sentiment and engagement around specific types of violent incidents.
• To develop a scalable method for automatic classification of security-related tweets based on latent topics.
• To detect underreported violent events that may not be captured by official police records.
• To support public policy and educational initiatives by understanding community reactions to different types of violence.

Proposed method

• Collected 258,848 security-related tweets from the Colombian news account @NoticiasRCN (2014–present).
• Generated dense vector representations of tweets using FastText word embeddings.
• Applied K-means clustering with topic count selected via LDA coherence (C_V) to optimize cluster granularity.
• Used Uniform Manifold Approximation and Projection (UMAP) for dimensionality reduction and visualization of tweet clusters.
• Manually labeled clusters by inspecting the 15 most representative tweets per cluster (closest to centroid).
• Re-evaluated clustering at 10 and 16 topics to determine optimal interpretability and topic separation.

Experimental results

Research questions

• RQ1 What are the most prominent sub-topics within security, violence, and crime-related news on Colombian Twitter?
• RQ2 How can latent topic modeling be fine-tuned to distinguish between closely related events such as traffic-related incidents?
• RQ3 Can a combination of FastText, K-means, and UMAP effectively identify interpretable and distinct event clusters in news tweets?
• RQ4 How does the number of clusters (10 vs. 16) affect topic interpretability and within-cluster dispersion?
• RQ5 To what extent can this method detect underreported violent events not captured by formal reporting systems?

Key findings

• The method successfully identified 10 interpretable topics, including attacks and aggressions, traffic and urban planning, thefts, traffic accidents, social problems, events in the traffic, violence against children and women, medical negligence, murders, and 'others'.
• The 10-topic configuration provided clearer topic separation and better interpretability than the 16-topic configuration, despite similar within-cluster dispersion.
• Traffic-related topics (traffic and urban planning, traffic accidents, events in the traffic) were well-differentiated, with distinct examples such as highway collapses and bus driver fights.
• The 'others' cluster contained diverse issues like animal abuse and government service failures, indicating the method's ability to capture heterogeneous but relevant security concerns.
• Clusters showed comparable within-cluster dispersion, indicating consistent coherence and stability across topics.
• The approach enables automatic classification of new security-related tweets and supports monitoring of vulnerable groups such as women, children, and crime victims.