[Paper Review] High-order Refining for End-to-end Chinese Semantic Role Labeling

Hao Fei, Yafeng Ren+1 more|arXiv (Cornell University)|Sep 15, 2020

Topic Modeling24 references4 citations

TL;DR

This paper proposes a high-order refining mechanism for end-to-end Chinese semantic role labeling (SRL) that enhances graph-based neural models by iteratively capturing global interactions between all predicate-argument pairs through attention-based feature refinement. The method improves performance on CoNLL09 and Universal Proposition Bank, achieving state-of-the-art results by mitigating long-range dependency issues in low-resource Chinese SRL.

ABSTRACT

Current end-to-end semantic role labeling is mostly accomplished via graph-based neural models. However, these all are first-order models, where each decision for detecting any predicate-argument pair is made in isolation with local features. In this paper, we present a high-order refining mechanism to perform interaction between all predicate-argument pairs. Based on the baseline graph model, our high-order refining module learns higher-order features between all candidate pairs via attention calculation, which are later used to update the original token representations. After several iterations of refinement, the underlying token representations can be enriched with globally interacted features. Our high-order model achieves state-of-the-art results on Chinese SRL data, including CoNLL09 and Universal Proposition Bank, meanwhile relieving the long-range dependency issues.

Motivation & Objective

• To address the limitations of first-order graph-based models in capturing long-range dependencies in Chinese semantic role labeling.
• To improve end-to-end SRL performance in low-resource settings like Chinese, where annotated data is limited.
• To develop a differentiable, parallelizable mechanism that enriches token representations through iterative global refinement.
• To achieve state-of-the-art results on Chinese SRL benchmarks by enhancing feature interactions across all predicate-argument pairs.

Proposed method

• The method builds on a baseline graph-based SRL model that uses BiLSTM encoders and biaffine attention to score all candidate predicate-argument pairs.
• A high-order refining module computes attention-based higher-order features across all predicate-argument pairs in each refinement iteration.
• These higher-order features are used to update the original token representations, enriching them with globally contextualized information.
• The refinement process is repeated for N iterations, allowing progressive global interaction and consistency in predictions.
• The module is fully differentiable and parallelizable, enabling efficient training and inference.
• The approach is evaluated with contextualized embeddings (ELMo and BERT), showing consistent improvements across settings.

Experimental results

Research questions

• RQ1 Can iterative high-order feature refinement improve end-to-end Chinese SRL beyond first-order models that make isolated local decisions?
• RQ2 How does the proposed high-order refining mechanism affect performance on long-range argument dependencies in Chinese text?
• RQ3 To what extent can the method mitigate data scarcity issues in low-resource languages like Chinese for SRL?
• RQ4 Does the global interaction mechanism lead to better consistency and accuracy in both predicate detection and argument role labeling?
• RQ5 How do different numbers of refinement iterations impact model performance and convergence?

Key findings

• The proposed high-order model achieves a new state-of-the-art F1 score of 88.6% on CoNLL09 for joint predicate and argument prediction, outperforming previous end-to-end models.
• With BERT embeddings, the model reaches 90.3% F1 on CoNLL09, demonstrating strong performance gains in low-resource settings.
• On the Universal Proposition Bank (UPB), the model achieves 72.4% F1 for argument role labeling, a significant improvement over prior end-to-end methods.
• The model performs best with 2 refinement iterations for predicate detection and 4 for argument labeling, indicating task-specific optimal refinement depth.
• The method significantly reduces performance drop on long-distance arguments, especially when surface distance ≥7, showing improved robustness to long-range dependencies.
• The ablation study confirms that the high-order refining mechanism consistently improves both subtasks across multiple datasets and embedding types.