[Paper Review] TAR on Social Media: A Framework for Online Content Moderation

Eugene Yang, David Lewis+1 more|arXiv (Cornell University)|Aug 29, 2021

Hate Speech and Cyberbullying Detection48 references4 citations

TL;DR

This paper proposes adapting Technology-Assisted Review (TAR) workflows—originally developed for high-recall legal discovery—to online content moderation, demonstrating that active learning with relevance feedback and uncertainty sampling reduces moderation costs by 20% to 55% across two public datasets. The approach prioritizes high-value content for human review, significantly cutting labor costs while maintaining high recall for policy-violating content.

ABSTRACT

Content moderation (removing or limiting the distribution of posts based on their contents) is one tool social networks use to fight problems such as harassment and disinformation. Manually screening all content is usually impractical given the scale of social media data, and the need for nuanced human interpretations makes fully automated approaches infeasible. We consider content moderation from the perspective of technology-assisted review (TAR): a human-in-the-loop active learning approach developed for high recall retrieval problems in civil litigation and other fields. We show how TAR workflows, and a TAR cost model, can be adapted to the content moderation problem. We then demonstrate on two publicly available content moderation data sets that a TAR workflow can reduce moderation costs by 20% to 55% across a variety of conditions.

Motivation & Objective

• To address the high cost and scalability challenges of human-only content moderation on social media platforms.
• To adapt Technology-Assisted Review (TAR) workflows—used in high-recall legal discovery—to the content moderation context.
• To evaluate whether active learning strategies like relevance feedback and uncertainty sampling reduce moderation costs compared to random sampling or full manual review.
• To analyze cost reductions across diverse content prevalence levels and batch sizes in real-world moderation scenarios.
• To explore the feasibility of transferring legal information retrieval techniques to social media content moderation.

Proposed method

• Adapting two-phase and one-phase TAR workflows from eDiscovery to content moderation, using human-in-the-loop active learning.
• Employing relevance feedback and uncertainty sampling to prioritize documents for human review, improving classifier performance iteratively.
• Applying a cost model that balances human review time and model training costs to optimize workflow efficiency.
• Using publicly available datasets (Wikipedia and ASKfm) to evaluate TAR performance across varying topic prevalence and batch sizes.
• Conducting 20 replicate runs per topic to ensure statistical robustness and computing precision and cost metrics across iterations.
• Visualizing precision trends via heatmaps to assess moderator experience and classifier quality over time.

Experimental results

Research questions

• RQ1 Can TAR workflows from legal eDiscovery be effectively adapted to online content moderation tasks?
• RQ2 To what extent do relevance feedback and uncertainty sampling reduce moderation costs compared to random sampling?
• RQ3 How does content prevalence (e.g., rare vs. frequent toxic content) affect cost reduction potential in TAR-based moderation?
• RQ4 Does the batch size used in active learning impact cost efficiency and classifier performance?
• RQ5 How does the precision of reviewed content evolve over iterations in different TAR strategies?

Key findings

• TAR workflows reduced moderation costs by 20% to 55% across both the Wikipedia and ASKfm datasets under various conditions.
• Relevance feedback and uncertainty sampling significantly outperformed random sampling, with cost reductions plateauing after 5,000 training documents for ASKfm and later for Wikipedia.
• For low-prevalence categories like 'Curse Exclusion' on ASKfm, cost savings reached up to 53.65% with uncertainty sampling after 8,002 iterations.
• Precision remained high and stable over time with relevance feedback, while uncertainty sampling maintained consistent precision across batches, indicating reliable classifier improvement.
• The cost of reviewing a random 80% sample was 92,590 (Wikipedia) and 90,958 (ASKfm), which was substantially higher than all TAR-based approaches.
• Larger batch sizes slightly reduced cost improvements due to less frequent retraining, suggesting smaller batches may be more cost-effective in practice.