[Paper Review] Neural Simile Recognition with Cyclic Multitask Learning and Local Attention

Jiali Zeng, Linfeng Song+4 more|arXiv (Cornell University)|Dec 19, 2019

Topic Modeling33 references4 citations

TL;DR

This paper proposes a cyclic multitask learning framework with local attention for neural simile recognition, enhancing inter-task dependencies by iteratively feeding outputs between simile sentence classification and component extraction. The method significantly outperforms state-of-the-art models, especially with BERT, by improving attention focus and modeling bidirectional task interactions through cyclic information flow.

ABSTRACT

Simile recognition is to detect simile sentences and to extract simile components, i.e., tenors and vehicles. It involves two subtasks: {\it simile sentence classification} and {\it simile component extraction}. Recent work has shown that standard multitask learning is effective for Chinese simile recognition, but it is still uncertain whether the mutual effects between the subtasks have been well captured by simple parameter sharing. We propose a novel cyclic multitask learning framework for neural simile recognition, which stacks the subtasks and makes them into a loop by connecting the last to the first. It iteratively performs each subtask, taking the outputs of the previous subtask as additional inputs to the current one, so that the interdependence between the subtasks can be better explored. Extensive experiments show that our framework significantly outperforms the current state-of-the-art model and our carefully designed baselines, and the gains are still remarkable using BERT.

Motivation & Objective

• To address the limitations of standard multitask learning in capturing bidirectional dependencies between simile recognition subtasks.
• To reduce attention errors caused by global attention mechanisms by focusing on local contexts around comparator words.
• To improve the accuracy of simile component extraction and sentence classification through iterative, cyclic information flow between tasks.
• To explore whether cyclic information propagation can better model the mutual dependencies between simile classification and component extraction than standard or decoding-based MTL.
• To evaluate the effectiveness of the framework on Chinese simile recognition, particularly with BERT-based encoders.

Proposed method

• The framework organizes simile sentence classification and component extraction into a cycle, where outputs from one task are fed as inputs to the next in a sequential loop.
• A local attention mechanism is applied to word representations centered on the comparator word, improving focus on relevant context and reducing noise from distant words.
• Word representations are concatenated with attention weights from the local attention layer before being passed to a CRF for sequence labeling in component extraction.
• The model uses a sentence decoder to reconstruct the input sentence, with decoder states combined with word representations to inform the next classification step.
• The cyclic design allows each subtask to benefit iteratively from the outputs of all others over K iterations, enhancing inter-task correlation modeling.
• The framework is trained end-to-end with shared bi-LSTM encoders and jointly optimized objectives for classification and extraction.

Experimental results

Research questions

• RQ1 Can cyclic information flow between simile recognition subtasks improve performance compared to standard multitask learning?
• RQ2 Does local attention around the comparator word reduce attention errors and improve focus on relevant simile components?
• RQ3 How does the proposed framework perform relative to state-of-the-art models, especially when fine-tuned with BERT?
• RQ4 To what extent do the mutual dependencies between simile classification and component extraction enhance overall recognition accuracy?
• RQ5 Can the cyclic MTL design be generalized to other NLP tasks with strong subtask interdependencies?

Key findings

• The proposed cyclic multitask learning framework achieves significant performance gains over the current state-of-the-art model on the Chinese simile recognition benchmark.
• The model outperforms standard multitask learning and carefully designed baselines, with gains remaining substantial even when using BERT as the backbone encoder.
• Local attention reduces attention interference from irrelevant words, especially those far from the comparator, by focusing on a constrained local window.
• The cyclic design enables better modeling of bidirectional dependencies between simile classification and component extraction, as evidenced by improved F1 scores on both tasks.
• The ablation study confirms that both cyclic information flow and local attention contribute independently and synergistically to performance improvement.
• The framework demonstrates strong generalization potential, with promising implications for other multitask NLP tasks such as aspect-level sentiment analysis.