[Paper Review] Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data

Tim Jansen, Yangling Tong+2 more|arXiv (Cornell University)|Dec 20, 2022

Topic Modeling4 citations

TL;DR

This paper proposes a perplexity-based method for detecting adult and harmful content in multilingual web data by training a language model exclusively on harmful texts and identifying high-perplexity documents as non-harmful. The approach achieves higher precision and a more realistic label distribution than traditional classifiers, especially on large-scale, noisy, multilingual corpora like OSCAR.

ABSTRACT

As demand for large corpora increases with the size of current state-of-the-art language models, using web data as the main part of the pre-training corpus for these models has become a ubiquitous practice. This, in turn, has introduced an important challenge for NLP practitioners, as they are now confronted with the task of developing highly optimized models and pipelines for pre-processing large quantities of textual data, which implies, effectively classifying and filtering multilingual, heterogeneous and noisy data, at web scale. One of the main components of this pre-processing step for the pre-training corpora of large language models, is the removal of adult and harmful content. In this paper we explore different methods for detecting adult and harmful of content in multilingual heterogeneous web data. We first show how traditional methods in harmful content detection, that seemingly perform quite well in small and specialized datasets quickly break down when confronted with heterogeneous noisy web data. We then resort to using a perplexity based approach but with a twist: Instead of using a so-called "clean" corpus to train a small language model and then use perplexity so select the documents with low perplexity, i.e., the documents that resemble this so-called "clean" corpus the most. We train solely with adult and harmful textual data, and then select the documents having a perplexity value above a given threshold. This approach will virtually cluster our documents into two distinct groups, which will greatly facilitate the choice of the threshold for the perplexity and will also allow us to obtain higher precision than with the traditional classification methods for detecting adult and harmful content.

Motivation & Objective

• To address the challenge of filtering adult and harmful content from large-scale, multilingual, and noisy web corpora such as OSCAR.
• To overcome the limitations of traditional classification models that fail to generalize on heterogeneous, noisy web data.
• To develop a scalable and efficient method for identifying harmful content that maintains high precision and realistic prediction distributions.
• To explore the feasibility of using perplexity as a proxy for content quality in a zero-shot, self-supervised manner on multilingual data.
• To enable practical deployment of harmful content detection at web scale using lightweight language models and fast inference.

Proposed method

• Train a lightweight language model (KenLM) exclusively on adult and harmful content from the OSCAR corpus to learn the linguistic patterns of such texts.
• Compute perplexity scores for each document in the test set using the trained model, treating high perplexity as an indicator of non-harmful content.
• Apply a threshold-based selection: documents with perplexity above the threshold are classified as non-harmful, while those below are flagged as potentially harmful.
• Leverage the inherent clustering of perplexity scores into two distinct groups (harmful vs. non-harmful) to simplify threshold selection and improve precision.
• Use FastText for baseline comparison in the first two approaches, focusing on speed and performance on large-scale data.
• Optimize inference speed by using small, efficient language models, enabling processing of large multilingual corpora (e.g., 84 hours for top 5 languages on modest hardware).

Experimental results

Research questions

• RQ1 Can a perplexity-based method trained solely on harmful data outperform traditional classification models in detecting adult and harmful content in noisy, multilingual web data?
• RQ2 How does the distribution of predicted harmful labels compare between traditional classifiers and the perplexity-based approach in large-scale corpora?
• RQ3 To what extent does training a language model on a representative subset of harmful data lead to generalization across the full OSCAR corpus?
• RQ4 Can the perplexity-based method achieve high precision and realistic label distributions without requiring large, manually annotated datasets?
• RQ5 How scalable and efficient is the perplexity-based method for processing multilingual web data at scale?

Key findings

• The perplexity-based method achieved higher precision than traditional classification models, which often produced unrealistic label distributions on noisy, heterogeneous data.
• The method naturally clustered documents into two distinct groups based on perplexity—harmful and non-harmful—enabling more reliable and intuitive threshold selection.
• Training the language model exclusively on harmful content allowed it to learn the linguistic patterns of such texts effectively, even with limited data.
• The approach demonstrated strong scalability, with estimated processing time of 84 hours for the five largest OSCAR languages on modest hardware.
• The method identified a realistic number of new harmful texts not previously labeled, aligning with findings from prior studies (e.g., Kreutzer et al., 2022).
• A limitation was the model’s failure to detect harmful content in parts of the OSCAR corpus not included in the training data, likely due to sequential data construction and linguistic diversity across files.