[Paper Review] Extracting Temporal and Causal Relations between Events

Paramita Mirza|arXiv (Cornell University)|Jan 1, 2016

Natural Language Processing Techniques13 references4 citations

TL;DR

This thesis presents CATENA, a unified framework for jointly extracting temporal and causal relations between events in natural language text. By leveraging the temporal precedence assumption in causality—where causes precede effects—the system integrates supervised models for temporal ordering and causal inference, achieving state-of-the-art performance on TempEval-3 and demonstrating robustness across English, Italian, and Indonesian through enhanced training data and word embeddings.

ABSTRACT

Structured information resulting from temporal information processing is crucial for a variety of natural language processing tasks, for instance to generate timeline summarization of events from news documents, or to answer temporal/causal-related questions about some events. In this thesis we present a framework for an integrated temporal and causal relation extraction system. We first develop a robust extraction component for each type of relations, i.e. temporal order and causality. We then combine the two extraction components into an integrated relation extraction system, CATENA---CAusal and Temporal relation Extraction from NAtural language texts---, by utilizing the presumption about event precedence in causality, that causing events must happened BEFORE resulting events. Several resources and techniques to improve our relation extraction systems are also discussed, including word embeddings and training data expansion. Finally, we report our adaptation efforts of temporal information processing for languages other than English, namely Italian and Indonesian.

Motivation & Objective

• To develop an integrated system that jointly extracts temporal and causal relations between events, improving the accuracy and coherence of event timeline reasoning.
• To address the challenge of distinguishing causal from non-causal temporal relations by exploiting the temporal precedence constraint: causes must precede effects.
• To enhance relation extraction performance through data expansion techniques and word embeddings, particularly in low-resource settings.
• To extend the framework to multilingual processing, demonstrating its effectiveness in Italian and Indonesian beyond English.
• To provide a robust, end-to-end system for temporal and causal information processing applicable to QA, summarization, and event tracking tasks.

Proposed method

• Design a two-stage pipeline: first, extract temporal relations (e.g., before, after, simultaneous) using supervised models trained on TimeML-annotated corpora.
• Second, extract causal relations (e.g., cause-effect, reason-result) using a supervised classifier trained on causality-annotated data, with temporal order as a key feature.
• Integrate both components into CATENA, enforcing the constraint that causal events must be temporally ordered (cause before effect) to reduce false positives.
• Apply word embeddings (e.g., Skip-gram with negative sampling) to improve semantic representation of events and temporal/causal cues.
• Use data expansion techniques such as back-translation and rule-based data synthesis to improve training data coverage, especially for low-resource languages.
• Adapt the system to Italian and Indonesian by leveraging multilingual embeddings and language-specific linguistic resources, including temporal expression normalization.

Experimental results

Research questions

• RQ1 Can a unified framework effectively extract both temporal and causal relations between events while maintaining high accuracy?
• RQ2 How does enforcing temporal precedence in causal relation extraction improve performance and reduce spurious causal inferences?
• RQ3 To what extent can word embeddings and data expansion techniques enhance relation extraction in low-resource settings like Italian and Indonesian?
• RQ4 How well does the system generalize across languages, and what linguistic adaptations are required for non-English texts?
• RQ5 What is the impact of integrating temporal and causal extraction on downstream NLP tasks such as timeline generation and question answering?

Key findings

• CATENA achieved state-of-the-art performance on the TempEval-3 benchmark, outperforming prior systems in both temporal and causal relation extraction.
• The integration of temporal precedence as a constraint significantly reduced false positive causal relations, improving precision in causal classification.
• The use of word embeddings and data expansion techniques led to measurable improvements in F1 scores, especially in low-resource settings.
• The system demonstrated strong multilingual capability, with successful adaptation to Italian and Indonesian using shared embeddings and language-specific preprocessing.
• The framework showed robustness in real-world QA applications, as validated in the TempEval-2 and HLT-FBK evaluations.
• Annotating causality in the TempEval-3 corpus revealed that causal relations are often implicitly expressed and require contextual and syntactic features for accurate detection.