[Paper Review] Chinese song iambics generation with neural attention-based model

Qixin Wang, Tianyi Luo+2 more|arXiv (Cornell University)|Jul 9, 2016

Topic Modeling17 references54 citations

TL;DR

This paper proposes an attention-based sequence-to-sequence model using bidirectional LSTMs to generate Chinese Song iambics, a poetic form with strict rhythmic and structural constraints. By integrating global context and employing hybrid style training, the model successfully learns complex patterns, achieving high-quality, rhythmically accurate poem generation in automatic and human evaluations.

ABSTRACT

Learning and generating Chinese poems is a charming yet challenging task. Traditional approaches involve various language modeling and machine translation techniques, however, they perform not as well when generating poems with complex pattern constraints, for example Song iambics, a famous type of poems that involve variable-length sentences and strict rhythmic patterns. This paper applies the attention-based sequence-to-sequence model to generate Chinese Song iambics. Specifically, we encode the cue sentences by a bi-directional Long-Short Term Memory (LSTM) model and then predict the entire iambic with the information provided by the encoder, in the form of an attention-based LSTM that can regularize the generation process by the fine structure of the input cues. Several techniques are investigated to improve the model, including global context integration, hybrid style training, character vector initialization and adaptation. Both the automatic and subjective evaluation results show that our model indeed can learn the complex structural and rhythmic patterns of Song iambics, and the generation is rather successful.

Motivation & Objective

• To address the challenge of generating Chinese Song iambics, which require strict rhythmic and structural patterns beyond the capability of traditional language models.
• To improve poem generation quality by incorporating fine-grained structural cues from input sentences using attention mechanisms.
• To explore techniques such as global context integration and hybrid style training to enhance structural fidelity and fluency in generated iambics.
• To evaluate the model’s ability to maintain rhythmic consistency and poetic coherence in long, variable-length poetic forms.

Proposed method

• Employing a bidirectional LSTM encoder to capture contextual representations from cue sentences provided as input.
• Using an attention-based LSTM decoder that attends to relevant encoder states to generate iambics token-by-token while respecting structural constraints.
• Integrating global context information to stabilize long-form generation and improve coherence across variable-length lines.
• Applying hybrid style training to jointly optimize for both fluency and structural accuracy in the generated poems.
• Initializing character embeddings with learned vectors and adapting them during training to better represent the poetic lexicon.
• Regularizing the generation process via attention mechanisms that focus on salient structural features in the input cues.

Experimental results

Research questions

• RQ1 Can an attention-based sequence-to-sequence model effectively learn and generate Chinese Song iambics with their complex rhythmic and structural constraints?
• RQ2 How does global context integration improve the coherence and structural consistency of generated iambics?
• RQ3 To what extent does hybrid style training enhance both fluency and structural fidelity in poem generation?
• RQ4 How do character vector initialization and adaptation affect the model’s ability to generate linguistically and rhythmically accurate poems?

Key findings

• The model successfully learns and reproduces the intricate rhythmic and structural patterns inherent in Chinese Song iambics.
• Automatic evaluation results demonstrate improved structural and linguistic quality compared to baseline models.
• Subjective evaluations confirm that the generated iambics are perceived as coherent, fluent, and rhythmically accurate by human annotators.
• Techniques such as global context integration and hybrid style training significantly enhance the model’s ability to maintain long-form coherence.
• Character vector initialization and adaptation contribute to better representation of poetic vocabulary and improved generation quality.
• The attention mechanism enables effective alignment between input cues and generated poetic lines, ensuring structural relevance.