[Paper Review] GCDT: A Chinese RST Treebank for Multigenre and Multilingual Discourse Parsing

Siyao Peng, Yang Janet Liu+1 more|arXiv (Cornell University)|Oct 19, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper introduces GCDT, the largest Chinese RST treebank with over 60K tokens across five genres, using the same relation inventory as English RST corpora. It achieves state-of-the-art performance in monolingual and multilingual RST parsing by leveraging cross-lingual training with multilingual embeddings and automatic EDU translation, with finetuning improving results on the smaller Chinese dataset and multilingual training yielding the best overall performance.

ABSTRACT

A lack of large-scale human-annotated data has hampered the hierarchical discourse parsing of Chinese. In this paper, we present GCDT, the largest hierarchical discourse treebank for Mandarin Chinese in the framework of Rhetorical Structure Theory (RST). GCDT covers over 60K tokens across five genres of freely available text, using the same relation inventory as contemporary RST treebanks for English. We also report on this dataset's parsing experiments, including state-of-the-art (SOTA) scores for Chinese RST parsing and RST parsing on the English GUM dataset, using cross-lingual training in Chinese and English with multilingual embeddings.

Motivation & Objective

• To address the lack of large-scale, human-annotated RST treebanks for Mandarin Chinese, which hinders hierarchical discourse parsing.
• To establish a high-quality, multi-genre RST corpus in Chinese that aligns with existing English RST standards for cross-lingual compatibility.
• To evaluate monolingual and multilingual discourse parsing performance on Chinese and English datasets using shared annotation frameworks.
• To investigate the effectiveness of multilingual training, finetuning, and automatic translation in improving parsing performance across low-resource and high-resource languages.
• To analyze genre-specific performance variations in discourse parsing to understand model generalization across text types.

Proposed method

• The authors constructed GCDT, a 60K-token, five-genre RST treebank for Mandarin Chinese, using consistent RST annotation guidelines and relation inventory as English RST corpora.
• They performed monolingual training using Chinese RoBERTa embeddings on GCDT, achieving SOTA results on the Chinese dataset.
• They conducted multilingual training by combining GCDT with the English GUM corpus, using XLM-RoBERTa-base as the multilingual embedding model.
• They applied finetuning on the smaller Chinese GCDT dataset after joint training with English data, improving performance on Chinese but not on English.
• They used automatic EDU-wise translation (via Google Translate) to augment training data, enabling the use of higher-performing monolingual embeddings.
• They evaluated performance using standard RST parsing metrics, including F1 scores on relation and span levels, across multiple test scenarios and genres.

Figure 1: A RST subtree with two relative clauses annotated as elaboration-attribute and same-unit in GCDT_academic_dingzhen with automatic $zh\rightarrow en$ translations appended after the source Chinese texts.

Experimental results

Research questions

• RQ1 Can a large-scale, multi-genre RST treebank for Mandarin Chinese be constructed with consistent annotation guidelines and relation inventory comparable to English RST corpora?
• RQ2 Does joint training on multilingual data (Chinese and English) improve discourse parsing performance on both languages compared to monolingual training?
• RQ3 Does finetuning on the smaller Chinese dataset (GCDT) yield better performance than joint training alone, and does it generalize to larger English datasets?
• RQ4 Can automatic EDU-wise translation of English discourse units improve parsing performance on Chinese, especially when combined with monolingual embeddings?
• RQ5 How do parsing performance and human-annotator agreement vary across different text genres in Chinese discourse parsing?

Key findings

• Monolingual RoBERTa embeddings outperformed multilingual embeddings in monolingual training, with hfl/chinese-roberta-wwm-ext achieving 51.76 F_Rel on GCDT.
• Joint training on GCDT and GUM-12 (12 genres) achieved the highest overall performance, with an F_Rel of 52.61 on GCDT, surpassing monolingual training.
• Finetuning on GCDT improved performance on the smaller Chinese dataset, increasing F_Rel from 50.45 to 52.61, but harmed performance on the larger English GUM dataset.
• Automatic EDU-wise translation combined with monolingual embeddings achieved the best performance in three out of four test scenarios, demonstrating its effectiveness for low-resource language parsing.
• Genre-wise analysis revealed that how-to guides (whow) were parsed most accurately, while performance was farthest from human ceiling scores in this genre, indicating genre-specific challenges.
• Performance varied significantly across genres, with academic articles being more challenging than how-to guides, suggesting that genre characteristics heavily influence parsing difficulty.

Figure 2: A RST subtree with attribution-positive scoping over causal-cause from GCDT_academic_dingzhen with automatic $zh\rightarrow en$ translation.