[Paper Review] Acquiring Annotated Data with Cross-lingual Explicitation for Implicit Discourse Relation Classification

Wei Shi, Frances Yung+1 more|arXiv (Cornell University)|Aug 30, 2018

Natural Language Processing Techniques28 references4 citations

TL;DR

This paper proposes a cross-lingual explicitation method to acquire high-quality annotated data for implicit discourse relation classification by leveraging human translators' insertion of connectives in multilingual translations. By back-translating from French, German, and Czech, and using majority voting across languages, the method significantly improves classification performance over baselines, with two-language agreement yielding the best results.

ABSTRACT

Implicit discourse relation classification is one of the most challenging and important tasks in discourse parsing, due to the lack of connective as strong linguistic cues. A principle bottleneck to further improvement is the shortage of training data (ca.~16k instances in the PDTB). Shi et al. (2017) proposed to acquire additional data by exploiting connectives in translation: human translators mark discourse relations which are implicit in the source language explicitly in the translation. Using back-translations of such explicitated connectives improves discourse relation parsing performance. This paper addresses the open question of whether the choice of the translation language matters, and whether multiple translations into different languages can be effectively used to improve the quality of the additional data.

Motivation & Objective

• Address the scarcity of annotated training data for implicit discourse relation classification, a major bottleneck in discourse parsing.
• Investigate whether the choice of translation language affects the quality and reliability of acquired training data.
• Explore whether combining explicitations from multiple translation languages improves data quality and downstream classification performance.
• Develop a method to identify and label implicit discourse relations using back-translations from parallel corpora with document-level alignment.
• Evaluate the effectiveness of majority voting across multiple translation languages to filter noisy or inconsistent explicitations.

Proposed method

• Utilize document-aligned parallel corpora (EuroParl) to identify sentence pairs where human translators inserted discourse connectives in the target language, even when the original English was implicit.
• Back-translate the translated sentences from French, German, and Czech back into English to recover explicit discourse markers.
• Apply a discourse relation parser (DRP) to the back-translated texts to identify and label the discourse relations.
• Use majority voting across multiple translation languages to select only those instances where at least two back-translations agree on the same relation type.
• Filter and retain only sentence pairs with consistent explicitations across languages to improve label reliability.
• Integrate the resulting high-quality, automatically labeled implicit discourse relation instances into the training data for the PDTB benchmark.

Experimental results

Research questions

• RQ1 Does the choice of translation language influence the quality and distribution of acquired implicit discourse relation instances?
• RQ2 Can combining explicitations from multiple translation languages yield higher-quality training data than using a single language?
• RQ3 Does majority voting across multiple back-translations reduce noise and improve label reliability?
• RQ4 How does the use of multilingual explicitation impact the performance of implicit discourse relation classification models?
• RQ5 Can the method detect cases where multiple discourse relations co-occur between the same argument pairs?

Key findings

• Instances where at least two back-translations agreed on the same discourse relation type yielded significantly higher classification performance than baselines without additional data or with data from a single language.
• The method achieved substantial performance improvements on the PDTB benchmark, demonstrating the effectiveness of multilingual explicitation for data acquisition.
• German translations showed a higher frequency of explicitations for conjunction relations, reflecting language-specific discourse marking tendencies.
• The approach successfully identified cases with multiple co-occurring discourse relations between the same arguments, highlighting its sensitivity to complex discourse structures.
• The qualitative analysis confirmed that consistent translation behavior across languages strengthens the method’s ability to learn reliable discourse signals and alternative lexicalizations.
• The use of majority voting significantly reduced noisy or inconsistent labels, improving the reliability of the acquired training data.