[Paper Review] Multi-Figurative Language Generation

Huiyuan Lai, Malvina Nissim|arXiv (Cornell University)|Sep 5, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces mFLAG, a multi-figurative language generation framework that jointly models five figurative forms—hyperbole, idiom, sarcasm, metaphor, and simile—using a two-stage training approach: multi-figurative pre-training with paraphrase data and a novel encoder injection mechanism to guide target figurative form generation without requiring parallel figurative-figurative pairs. The model outperforms strong baselines and enables zero-shot figurative-to-figurative generation by leveraging cross-figurative knowledge transfer.

ABSTRACT

Figurative language generation is the task of reformulating a given text in the desired figure of speech while still being faithful to the original context. We take the first step towards multi-figurative language modelling by providing a benchmark for the automatic generation of five common figurative forms in English. We train mFLAG employing a scheme for multi-figurative language pre-training on top of BART, and a mechanism for injecting the target figurative information into the encoder; this enables the generation of text with the target figurative form from another figurative form without parallel figurative-figurative sentence pairs. Our approach outperforms all strong baselines. We also offer some qualitative analysis and reflections on the relationship between the different figures of speech.

Motivation & Objective

• To address the lack of joint modeling of multiple figurative language forms in neural generation, which currently requires separate models per figure of speech.
• To enable zero-shot generation between different figurative forms (e.g., hyperbole to metaphor) without parallel figurative-figurative sentence pairs.
• To explore cross-figurative knowledge transfer by jointly pre-training on multiple figurative forms using paraphrase data and form-specific labels.
• To design a mechanism that injects target figurative information into the encoder to guide generation without relying on parallel data.
• To evaluate the feasibility and characteristics of direct figurative-to-figurative generation versus figurative-literal-figurative pivoting.

Proposed method

• A two-stage training scheme: first, multi-figurative pre-training on a mixture of literal and figurative sentences with form-specific tokens prefixed to each input.
• Second, supervised fine-tuning using parallel literal-figurative pairs across all five figurative forms to enable end-to-end generation.
• A novel encoder injection mechanism that projects the target figurative form label into the input embedding space, enabling the model to attend to the desired figurative form during decoding.
• The use of paraphrase data during pre-training to improve contextual fidelity and form strength across different figurative forms.
• Comparison of direct figurative-to-figurative generation with a pivot-based method using literal forms as intermediaries to assess performance and linguistic coherence.
• Visual analysis via PCA on encoder representations to examine semantic clustering of tokens across different figurative forms.

Experimental results

Research questions

• RQ1 Can a single neural model effectively generate multiple figurative language forms from literal and figurative inputs without requiring parallel figurative-figurative data?
• RQ2 To what extent can cross-figurative knowledge transfer improve performance in multi-figurative language generation?
• RQ3 How does the proposed encoder injection mechanism affect the model's ability to generate the target figurative form while preserving context?
• RQ4 What are the linguistic similarities and differences between different figurative forms, as revealed by classifier generalization across forms?
• RQ5 Is direct figurative-to-figurative generation superior to using literal forms as pivots in terms of coherence and form accuracy?

Key findings

• The mFLAG model outperforms all strong baselines in multi-figurative language generation, demonstrating the effectiveness of joint modeling and cross-figurative knowledge transfer.
• The encoder injection mechanism enables successful figurative-to-figurative generation without parallel figurative-figurative pairs, achieving high performance on unseen form transitions.
• PCA visualization shows that token representations for the target figurative form (e.g., 'on pins and needles' in hyperbole) cluster more closely in the embedding space when the injection mechanism is used, indicating better alignment with target form semantics.
• Classifiers trained on one figurative form generalize moderately well to others—especially hyperbole and idiom to simile (F1 > 0.79)—suggesting shared non-literal features across forms.
• The simile classifier generalizes poorly to other forms (F1 < 0.11), likely due to its distinct syntactic format 'like a X', which is easily detectable and limits cross-form transfer.
• The overall literal vs. figurative classifier achieves F1 scores above 0.69 across all forms, confirming the feasibility of multi-figurative modeling.