[Paper Review] Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings

Dorottya Demszky, Nikhil Garg+5 more|arXiv (Cornell University)|Apr 2, 2019

Social Media and Politics53 references46 citations

TL;DR

The paper presents an NLP framework to analyze four linguistic dimensions of polarization in social media—topic choice, framing, affect, and illocutionary force—applied to 4.4 million tweets about 21 mass shootings.

ABSTRACT

We provide an NLP framework to uncover four linguistic dimensions of political polarization in social media: topic choice, framing, affect and illocutionary force. We quantify these aspects with existing lexical methods, and propose clustering of tweet embeddings as a means to identify salient topics for analysis across events; human evaluations show that our approach generates more cohesive topics than traditional LDA-based models. We apply our methods to study 4.4M tweets on 21 mass shootings. We provide evidence that the discussion of these events is highly polarized politically and that this polarization is primarily driven by partisan differences in framing rather than topic choice. We identify framing devices, such as grounding and the contrasting use of the terms "terrorist" and "crazy", that contribute to polarization. Results pertaining to topic choice, affect and illocutionary force suggest that Republicans focus more on the shooter and event-specific facts (news) while Democrats focus more on the victims and call for policy changes. Our work contributes to a deeper understanding of the way group divisions manifest in language and to computational methods for studying them.

Motivation & Objective

• Motivate understanding of how language expresses political polarization on social media.
• Develop a comprehensive framework to analyze polarization across multiple linguistic dimensions (topic, framing, affect, illocutionary force).
• Quantify polarization and its evolution within and across mass shooting events using Twitter data.
• Identify how shooter race interacts with framing and topic choice in polarized discourse.
• Provide datasets and methods to enable replication and further study of linguistic polarization.

Proposed method

• Define vocabularies and token-based features for each event and compute leave-out partisanship to measure language polarization.
• Develop an embedding-based tweet clustering approach to induce cohesive, event-independent topics and compare against MALLET and Biterm Topic Model (BTM).
• Apply Arora et al. (2017) sentence embeddings on GloVe-based token representations to cluster tweets via k-means (cosine distance).
• Compute within-topic and between-topic partisanship using the leave-out estimator on topic-labeled data.
• Decompose polarization into topic-level and within-topic components to assess whether polarization is driven by topic choices or framing within topics.
• Analyze effects of shooter race on framing and topic preferences, using partisan log odds ratios and context grounding.

Experimental results

Research questions

• RQ1 How polarized are Twitter discussions about mass shootings along partisan lines?
• RQ2 To what extent does polarization arise from topic choice versus framing within topics?
• RQ3 How does the shooter’s race influence framing, topic preferences, and affective expressions?
• RQ4 What are the specific framing devices and illocutionary forces that characterize partisan discourse in this domain?
• RQ5 Do affect and modality (illocutionary force) contribute to partisan polarization in these events?

Key findings

• Tweets about mass shootings are highly polarized, with leave-out partisanship values ranging roughly from .517 to .547 across events.
• Within-topic polarization increases over time, while between-topic polarization remains stable.
• Republicans focus more on shooter identity and news; Democrats focus more on victims and policy changes.
• Framing devices such as grounding and differential use of terms like “terrorist” and “crazy” depend on shooter race, with Democrats more likely to label white shooters as terrorists and Republicans more likely to label non-white shooters similarly in different patterns.
• Affect analysis shows Democrats expressing more positive sentiment, sadness, and trust, while Republicans express more fear and disgust, particularly when the shooter is a person of color.
• Modals (should, must, have to, need to) are used predominantly to call for action, with Democrats more likely to employ them across topics.