[Paper Review] Pun Generation with Surprise

He He, Nanyun Peng+1 more|arXiv (Cornell University)|Apr 15, 2019

Humor Studies and Applications29 references4 citations

TL;DR

This paper proposes SurGen, an unsupervised pun generation method based on the local-global surprisal principle, which exploits contrasting surprise levels between a pun word and its homophone alternative in local versus global sentence contexts. By using a retrieve-and-edit framework with a language model and skip-gram word suggestions, SurGen achieves a 31% success rate in human evaluation—tripling the 9% success of a neural generation baseline—demonstrating effective, novel pun generation without large annotated corpora.

ABSTRACT

We tackle the problem of generating a pun sentence given a pair of homophones (e.g., "died" and "dyed"). Supervised text generation is inappropriate due to the lack of a large corpus of puns, and even if such a corpus existed, mimicry is at odds with generating novel content. In this paper, we propose an unsupervised approach to pun generation using a corpus of unhumorous text and what we call the local-global surprisal principle: we posit that in a pun sentence, there is a strong association between the pun word (e.g., "dyed") and the distant context, as well as a strong association between the alternative word (e.g., "died") and the immediate context. This contrast creates surprise and thus humor. We instantiate this principle for pun generation in two ways: (i) as a measure based on the ratio of probabilities under a language model, and (ii) a retrieve-and-edit approach based on words suggested by a skip-gram model. Human evaluation shows that our retrieve-and-edit approach generates puns successfully 31% of the time, tripling the success rate of a neural generation baseline.

Motivation & Objective

• To address the lack of large-scale pun corpora and the limitations of supervised generation for creative content.
• To develop an unsupervised approach to pun generation that produces novel, humorous sentences rather than mimicking existing data.
• To formalize a principle of humor based on surprisal contrast between local and global context.
• To improve pun generation beyond neural sequence models by incorporating cognitive principles of surprise and incongruity.

Proposed method

• Propose the local-global surprisal principle: a pun word is locally surprising (unusual in immediate context) but globally expected (supported by broader context), while the alternative word shows the opposite.
• Define a surprisal metric as the log-likelihood ratio of the pun word versus the alternative word given local and global contexts: $ S(c) = -\log \frac{p(w^p|c)}{p(w^a|c)} $.
• Implement a retrieve-and-edit framework: retrieve candidate sentences from a non-humorous corpus using skip-gram embeddings, then edit them to insert the pun word.
• Use a neural language model to score and refine generated sentences for grammaticality and fluency.
• Operationalize the surprisal principle by selecting sentences where the pun word is locally surprising but globally plausible.
• Train a neural smoother to improve fluency and coherence of generated puns.

Experimental results

Research questions

• RQ1 Can surprisal contrast between local and global context serve as a reliable signal for identifying humorous puns?
• RQ2 Can the local-global surprisal principle be effectively operationalized to generate novel, creative puns without large annotated training data?
• RQ3 Does a retrieve-and-edit approach based on this principle outperform end-to-end neural generation in pun quality and success rate?
• RQ4 To what extent does the surprisal metric correlate with human judgments of pun humor?
• RQ5 Can the method generate puns that are both grammatically correct and perceived as funny by humans?

Key findings

• The local-global surprisal metric shows a strong correlation with human ratings of pun funniness, validating its cognitive plausibility.
• The SurGen retrieve-and-edit system achieves a 31% success rate in human evaluation, tripling the 9% success rate of a neural generation baseline.
• Human evaluators rated SurGen-generated puns as significantly funnier than those from the neural baseline.
• The method successfully generates puns by creating a contrast in surprise levels: the pun word is locally unexpected but globally plausible.
• The approach generalizes across 150 homophone pairs and demonstrates robustness without requiring fine-tuning on pun-specific data.
• The study highlights the challenge of distinguishing creative novelty from linguistic nonsense, suggesting that future work must better disentangle these concepts.