[Paper Review] Temporal Relation Extraction with a Graph-Based Deep Biaffine Attention Model

Bo-Ying Su, Shang-Ling Hsu+2 more|arXiv (Cornell University)|Jan 16, 2022

Topic Modeling4 citations

TL;DR

This paper proposes a graph-based deep biaffine attention model that jointly predicts event arcs and temporal relation labels in an end-to-end manner, eliminating the need for prior event annotation. By decoupling arc and relation prediction via separate biaffine modules, the model achieves state-of-the-art F1 scores on benchmark datasets while accelerating inference by 39% through fully parallelizable computation.

ABSTRACT

Temporal information extraction plays a critical role in natural language understanding. Previous systems have incorporated advanced neural language models and have successfully enhanced the accuracy of temporal information extraction tasks. However, these systems have two major shortcomings. First, they fail to make use of the two-sided nature of temporal relations in prediction. Second, they involve non-parallelizable pipelines in inference process that bring little performance gain. To this end, we propose a novel temporal information extraction model based on deep biaffine attention to extract temporal relationships between events in unstructured text efficiently and accurately. Our model is performant because we perform relation extraction tasks directly instead of considering event annotation as a prerequisite of relation extraction. Moreover, our architecture uses Multilayer Perceptrons (MLP) with biaffine attention to predict arcs and relation labels separately, improving relation detecting accuracy by exploiting the two-sided nature of temporal relationships. We experimentally demonstrate that our model achieves state-of-the-art performance in temporal relation extraction.

Motivation & Objective

• To address the inefficiency and accuracy limitations of existing two-stage temporal relation extraction pipelines.
• To improve performance by modeling the two-sided nature of temporal relations more effectively.
• To eliminate dependency on prior event annotation during inference, enabling true end-to-end relation extraction.
• To accelerate inference through a parallelizable architecture that avoids nested loops in relation prediction.
• To demonstrate that separating arc prediction from relation classification enhances detection of vague and non-relations (NONE/VAGUE).

Proposed method

• Uses a fine-tuned BERT model for contextual token embeddings.
• Employs a bidirectional LSTM to encode contextual representations of input sequences.
• Deploys two distinct biaffine attention modules: one for predicting arcs (presence of relation) and another for predicting relation labels.
• Integrates a lightweight decoder that combines top-scoring arc and relation predictions to construct the final temporal relation graph.
• Optimizes the model via multi-task learning using joint cross-entropy loss on both arc and relation prediction heads.
• Enables fully parallelized inference by decoupling arc and relation prediction, removing the need for sequential event pair enumeration.

Experimental results

Research questions

• RQ1 Can a unified, end-to-end model outperform two-stage approaches in temporal relation extraction without requiring gold event spans during inference?
• RQ2 Does separating arc prediction (existence of relation) from relation label classification improve performance, especially for NONE and VAGUE relations?
• RQ3 Can a biaffine attention mechanism enhance relation detection accuracy by exploiting the two-sided nature of temporal relations?
• RQ4 To what extent does a parallelizable architecture reduce inference time without sacrificing F1 score?
• RQ5 How does the proposed model compare to state-of-the-art baselines in terms of both accuracy and computational efficiency?

Key findings

• The model achieves a state-of-the-art F1 score of 50.9 on the TB-Dense dataset, outperforming prior SOTA baselines.
• Inference speed reaches 452 sentences per second on TB-Dense, a 39% improvement over prior methods.
• The ablation study shows that removing the dedicated ARC module reduces F1 by up to 1.5 points, especially harming VAGUE relation detection.
• The biaffine attention mechanism improves performance slightly over standard linear layers and runs faster on average.
• The model maintains high accuracy even when the ARC module is disabled, but performance drops significantly on VAGUE relations, proving the value of dedicated arc prediction.
• The proposed architecture enables fully parallelized decoding, eliminating the non-parallelizable nested loops used in prior systems.