[Paper Review] Opinion Conflicts: An Effective Route to Detect Incivility in Twitter

Suman Kalyan Maity, Aishik Chakraborty+2 more|arXiv (Cornell University)|Sep 2, 2018

Hate Speech and Cyberbullying Detection92 references16 citations

TL;DR

This paper proposes a deep learning model that detects incivility on Twitter by leveraging opinion conflicts between users and named entities, using a character-level CNN with entity-specific sentiment features, achieving 93.3% accuracy—4.9% higher than the best baseline.

ABSTRACT

In Twitter, there is a rising trend in abusive behavior which often leads to incivility. This trend is affecting users mentally and as a result they tend to leave Twitter and other such social networking sites thus depleting the active user base. In this paper, we study factors associated with incivility. We observe that the act of incivility is highly correlated with the opinion differences between the account holder (i.e., the user writing the incivil tweet) and the target (i.e., the user for whom the incivil tweet is meant for or targeted), toward a named entity. We introduce a character level CNN model and incorporate the entity-specific sentiment information for efficient incivility detection which significantly outperforms multiple baseline methods achieving an impressive accuracy of 93.3% (4.9% improvement over the best baseline). In a post-hoc analysis, we also study the behavioral aspects of the targets and account holders and try to understand the reasons behind the incivility incidents. Interestingly, we observe that there are strong signals of repetitions in incivil behavior. In particular, we find that there are a significant fraction of account holders who act as repeat offenders - attacking the targets even more than 10 times. Similarly, there are also targets who get targeted multiple times. In general, the targets are found to have higher reputation scores than the account holders.

Motivation & Objective

• To investigate the role of opinion conflicts between users and named entities in triggering incivility on Twitter.
• To develop an automated, scalable method for detecting incivility that outperforms existing baselines.
• To analyze behavioral patterns of incivility perpetrators and targets, including repetition and reputation dynamics.
• To provide actionable insights for platform design and policy to reduce online toxicity and improve community health.

Proposed method

• The authors propose a character-level convolutional neural network (CNN) model to capture fine-grained linguistic patterns in tweets.
• Entity-specific sentiment information is incorporated into the model to highlight opinion conflicts between the user (account holder) and the target entity.
• The model uses word embeddings and character-level LSTMs to encode tweet content, enhancing sensitivity to abusive language and emotional tone.
• Sentiment scores for named entities are extracted from the target user's historical tweets to quantify opinion conflict.
• The model is trained end-to-end on a dataset of labeled incivility tweets, with loss optimized via cross-entropy.
• Post-hoc behavioral analysis is conducted on user interaction patterns, including repetition frequency and reputation scores (e.g., follower count, engagement).

Experimental results

Research questions

• RQ1 To what extent are opinion conflicts between users and named entities predictive of incivility in tweets?
• RQ2 How does incorporating entity-specific sentiment improve incivility detection performance compared to baseline models?
• RQ3 What behavioral patterns—such as repetition or reputation—characterize incivility perpetrators and targets?
• RQ4 Can early detection of opinion conflicts help prevent escalation into incivility or cyberbullying?

Key findings

• The proposed model achieves 93.3% accuracy and an F1-score of 0.82, representing a 4.9% improvement over the best baseline in accuracy and a 6.5% improvement in F1-score.
• A significant proportion of account holders (repeat offenders) target the same individuals more than 10 times, indicating persistent incivility behavior.
• Targets of incivility consistently have higher reputation scores (e.g., more followers, higher engagement) than the account holders, contrasting with typical cyberbullying dynamics.
• Account holders' tweets contain a higher frequency of negative sentiment words, including 'swear', 'sex', 'religion', and 'death'—indicators of emotional intensity.
• Targets' tweets are rich in positive emotions, suggesting a contrast in emotional tone between perpetrators and victims.
• There is strong evidence of repeated targeting, with some individuals being attacked multiple times, highlighting the need for early intervention systems.