[Paper Review] Coarse-to-Fine Entity Representations for Document-level Relation Extraction

Damai Dai, Jing Ren+3 more|arXiv (Cornell University)|Dec 4, 2020

Topic Modeling45 references4 citations

TL;DR

This paper proposes CFER, a coarse-to-fine entity representation model for document-level relation extraction that first captures global contextual information via Densely Connected Graph Convolutional Networks (DCGCN) and then uses attention-guided path encoding to model long-distance entity interactions. The method achieves state-of-the-art performance on DocRED and CDR, demonstrating robustness to label imbalance and superior modeling of inter-sentence relations.

ABSTRACT

Document-level Relation Extraction (RE) requires extracting relations expressed within and across sentences. Recent works show that graph-based methods, usually constructing a document-level graph that captures document-aware interactions, can obtain useful entity representations thus helping tackle document-level RE. These methods either focus more on the entire graph, or pay more attention to a part of the graph, e.g., paths between the target entity pair. However, we find that document-level RE may benefit from focusing on both of them simultaneously. Therefore, to obtain more comprehensive entity representations, we propose the Coarse-to-Fine Entity Representation model (CFER) that adopts a coarse-to-fine strategy involving two phases. First, CFER uses graph neural networks to integrate global information in the entire graph at a coarse level. Next, CFER utilizes the global information as a guidance to selectively aggregate path information between the target entity pair at a fine level. In classification, we combine the entity representations from both two levels into more comprehensive representations for relation extraction. Experimental results on two document-level RE datasets, DocRED and CDR, show that CFER outperforms existing models and is robust to the uneven label distribution.

Motivation & Objective

• To address the limitations of existing graph-based models in document-level relation extraction, which either overemphasize global structure or local paths.
• To simultaneously capture global contextual information and model long-distance interactions between target entities in a document.
• To improve relation extraction performance on long documents with complex inter-sentence relations.
• To enhance robustness to uneven label distribution, especially for long-tail relations.
• To develop a unified framework that integrates both global and fine-grained path-level representations effectively.

Proposed method

• Construct a document-level graph with five edge types: syntactic dependencies, adjacent words, cross-sentence connections, coreference links, and self-loops.
• Apply Densely Connected Graph Convolutional Networks (DCGCN) to aggregate global contextual information across the entire graph in a coarse-level phase.
• Use an attention-based path encoding mechanism in a fine-level phase that is guided by the coarse-level representations to selectively aggregate path information between the target entity pair.
• Combine the coarse-level global representations and fine-level path-aware representations into comprehensive entity embeddings for relation classification.
• Train the model end-to-end using cross-entropy loss on relation prediction, with attention weights learned to focus on informative paths.
• Integrate both levels of representation via concatenation or element-wise combination before the final classifier.

Experimental results

Research questions

• RQ1 Can combining global graph-level context with targeted path-level information improve document-level relation extraction?
• RQ2 Does a coarse-to-fine strategy enhance modeling of long-distance entity interactions compared to single-phase approaches?
• RQ3 How does the proposed method perform under class imbalance, particularly for long-tail relations?
• RQ4 Can the coarse-level representation effectively guide the fine-level path aggregation to avoid suboptimal attention allocation?
• RQ5 To what extent does the model outperform existing graph-based and non-graph-based baselines on standard benchmarks?

Key findings

• CFER achieves an F1 score of 56.40 on the development set of DocRED, outperforming all ablated variants and existing models.
• The ablation study shows that removing the coarse-level DCGCN blocks reduces performance by 2.08 F1 points, highlighting its importance.
• The model improves micro-F1 by 4.12 points when including fine-level representations, confirming their value in capturing long-distance relations.
• CFER shows stronger performance gains on long-tail relations (distance [8, ∞)), with a micro-F1 of 61.5, indicating robustness to label imbalance.
• Case studies reveal that CFER (Full) learns balanced attention weights across multiple useful paths, while ablated models focus on suboptimal paths due to lack of global guidance.
• The model successfully extracts relational facts involving inter-sentence reasoning, such as the P140 relation, which only the full model can recover.