[Paper Review] "President Vows to Cut Hair": Dataset and Analysis of Creative Text Editing for Humorous Headlines

Nabil Hossain, John Krumm+1 more|arXiv (Cornell University)|Jun 1, 2019

Humor Studies and Applications27 references34 citations

TL;DR

The paper introduces Humicroedit, a dataset of 15,095 edited news headlines intended to be humorous, analyzes humor generation strategies, and builds baseline humor detection models.

ABSTRACT

We introduce, release, and analyze a new dataset, called Humicroedit, for research in computational humor. Our publicly available data consists of regular English news headlines paired with versions of the same headlines that contain simple replacement edits designed to make them funny. We carefully curated crowdsourced editors to create funny headlines and judges to score a to a total of 15,095 edited headlines, with five judges per headline. The simple edits, usually just a single word replacement, mean we can apply straightforward analysis techniques to determine what makes our edited headlines humorous. We show how the data support classic theories of humor, such as incongruity, superiority, and setup/punchline. Finally, we develop baseline classifiers that can predict whether or not an edited headline is funny, which is a first step toward automatically generating humorous headlines as an approach to creating topical humor.

Motivation & Objective

• Create a publicly available dataset of short, single-word edits to headlines that induce humor.
• Analyze how micro-edits relate to classic humor theories (incongruity, setup/punchline, superiority).
• Identify editing strategies and word clusters that maximize humor potential.
• Develop baseline classifiers to predict whether an edited headline is funny.
• Explore implications for automatic humorous headline generation and personalization.

Proposed method

• Collect original Reddit-based headlines and apply small, single-word edits (micro-edits) using trained editors.
• Annotate edited headlines with five judges to obtain mean funniness scores.
• Quantitatively analyze humor through theories and word-distance, setup/punchline positioning, and word cluster analyses.
• Train non-neural (logistic regression, random forest, SVM) and neural (bi-directional LSTM) baselines to predict funniness using edited headlines.
• Evaluate classifiers on varying data splits to assess extremes and overall performance.

Experimental results

Research questions

• RQ1 What types of micro-edits reliably produce humorous headlines?
• RQ2 How do humor theories (incongruity, setup/punchline, superiority) manifest in micro-edited headlines?
• RQ3 Can machine learning models predict whether an edited headline is funny based on the headline text alone?
• RQ4 How does replacing different parts of speech (noun/verb vs. entities) affect humor detection performance?
• RQ5 What are the limits and challenges in constructing a humor dataset from headlines and how do editors/judges impact reliability?

Key findings

• Humicroedit contains 15,095 edited headlines with five judges’ mean funniness scores.
• Longer headlines tend to show higher humor potential than very short ones.
• Incongruity between replaced and added words correlates with humor, especially for extreme funniness cases.
• Setup/punchline patterns emerge as editors prefer later-word substitutions that yield funnier outcomes.
• Clusters of replacement words reveal humor strategies across categories like clothes, sex, foods, and insults.
• Baseline LSTM models using GloVe embeddings achieve about 68.54% accuracy in funny/not-funny classification on certain splits, outperforming non-neural baselines.