[Paper Review] Domain Adaptative Causality Encoder

Farhad Moghimifar, Gholamreza Haffari+1 more|arXiv (Cornell University)|Nov 27, 2020

Topic Modeling41 references4 citations

TL;DR

This paper proposes a domain-adaptive causality encoder (ACE) that leverages dependency tree representations and adversarial learning to improve causal relation identification and localization across distributionally different domains. By introducing the MedCaus dataset and applying gradient reversal for domain adaptation, ACE achieves up to 7% improvement over state-of-the-art models on benchmark datasets.

ABSTRACT

Current approaches which are mainly based on the extraction of low-level relations among individual events are limited by the shortage of publicly available labelled data. Therefore, the resulting models perform poorly when applied to a distributionally different domain for which labelled data did not exist at the time of training. To overcome this limitation, in this paper, we leverage the characteristics of dependency trees and adversarial learning to address the tasks of adaptive causality identification and localisation. The term adaptive is used since the training and test data come from two distributionally different datasets, which to the best of our knowledge, this work is the first to address. Moreover, we present a new causality dataset, namely MedCaus, which integrates all types of causality in the text. Our experiments on four different benchmark causality datasets demonstrate the superiority of our approach over the existing baselines, by up to 7% improvement, on the tasks of identification and localisation of the causal relations from the text.

Motivation & Objective

• To address the challenge of domain shift in textual causality extraction, where models trained on one domain (e.g., medical) underperform on out-of-domain data (e.g., financial).
• To develop a neural model that jointly captures syntactic and semantic dependencies in sentences for more accurate causality identification and localization.
• To mitigate distributional differences between source and target domains using adversarial domain adaptation with gradient reversal.
• To create a comprehensive causality dataset, MedCaus, that includes explicit, implicit, nested, and non-causal relations for improved training and evaluation.

Proposed method

• The model uses graph convolutional networks (GCNs) to encode dependency trees, capturing both syntactic and semantic relations among words in a sentence.
• It applies a gradient reversal layer to align the feature distributions of source and target domains, minimizing domain shift during training.
• The adversarial training framework includes a domain discriminator that encourages the encoder to produce domain-invariant representations.
• The architecture is jointly optimized for causality identification (binary classification) and causality localization (span extraction) using multi-task learning.
• The MedCaus dataset, comprising 15,000 human-annotated sentences, is constructed from medical literature to cover all causality types: explicit, implicit, nested, and non-causal.
• Feature representations from the encoder are visualized using t-SNE to demonstrate successful domain alignment after applying the ACE module.

Experimental results

Research questions

• RQ1 Can a causality model trained on one domain generalize effectively to a distributionally different domain without retraining on labeled target data?
• RQ2 How effective is adversarial domain adaptation with gradient reversal in improving causality identification and localization across domains?
• RQ3 To what extent does incorporating dependency tree structures enhance causality detection compared to standard sequence models?
• RQ4 Can a unified model achieve superior performance on both causality identification and localization tasks through joint learning?

Key findings

• The proposed ACE model improves F1 scores by up to 7% over existing baselines on causality identification and localization tasks across four benchmark datasets.
• Domain adaptation via gradient reversal significantly enhances model generalization, with performance gains of up to 39% on target datasets when applied to baseline models.
• The ACE model achieves the highest recall and F1 scores among all compared models, outperforming CDAN in both metrics despite CDAN's higher precision.
• Qualitative analysis confirms that the model correctly identifies causal relations and accurately localizes cause-effect spans in diverse domains, including medical, financial, and general text.
• t-SNE visualization demonstrates that the ACE module successfully aligns the feature distributions of source and target domains, reducing domain shift.
• The MedCaus dataset effectively captures the full spectrum of causality types, including nested and implicit relations, enabling more robust training and evaluation.