[论文解读] Neural Metric Learning for Fast End-to-End Relation Extraction

Tung Tran, Ramakanth Kavuluru|arXiv (Cornell University)|May 17, 2019

Topic Modeling参考文献 41被引用 40

一句话总结

Introduce a table-based end-to-end relation extraction model that uses a relation-metric network with 2D convolutions to jointly learn NER and RE, achieving state-of-the-art results with much faster training/testing times.

ABSTRACT

Relation extraction (RE) is an indispensable information extraction task in several disciplines. RE models typically assume that named entity recognition (NER) is already performed in a previous step by another independent model. Several recent efforts, under the theme of end-to-end RE, seek to exploit inter-task correlations by modeling both NER and RE tasks jointly. Earlier work in this area commonly reduces the task to a table-filling problem wherein an additional expensive decoding step involving beam search is applied to obtain globally consistent cell labels. In efforts that do not employ table-filling, global optimization in the form of CRFs with Viterbi decoding for the NER component is still necessary for competitive performance. We introduce a novel neural architecture utilizing the table structure, based on repeated applications of 2D convolutions for pooling local dependency and metric-based features, that improves on the state-of-the-art without the need for global optimization. We validate our model on the ADE and CoNLL04 datasets for end-to-end RE and demonstrate $\approx 1\%$ gain (in F-score) over prior best results with training and testing times that are seven to ten times faster --- the latter highly advantageous for time-sensitive end user applications.

研究动机与目标

• Motivate end-to-end E2E RE by exploiting inter-task correlations between NER and RE.
• Propose a table-filled neural architecture that uses metric-based features and 2D convolutions to pool local and global information.
• Eliminate the need for expensive decoding (e.g., beam search) while improving accuracy.
• Demonstrate state-of-the-art performance and faster training/testing times on benchmark datasets (CoNLL04 and ADE).
• Provide interpretability insights through analysis of hidden pooling layers.

提出的方法

• Formulate RE as table filling over an n x n table with BILOU-encoded entities on the diagonal and relation tags on intersecting entity spans.
• Introduce a relation-metric network that learns a relatedness function s_R(h_i, h_j) = h_i^T R^k h_j across multiple metric channels (G tensor).
• Compute context embeddings via Bi-LSTM over word and character features.
• Incorporate dependency and position embeddings to enrich pairwise features before 3x3 2D convolutions.
• Apply iterative pooling with λ convolutional layers to propagate information and corroborate early decisions.
• Decode relations by aggregating Q scores over intersecting entity spans and selecting the argmax tag.

实验结果

研究问题

• RQ1 How can end-to-end NER and RE be effectively modeled in a unified architecture without relying on beam search or CRFs for decoding?
• RQ2 Can a table-based representation with metric-based features and local pooling capture inter-token relations efficiently and accurately?
• RQ3 Does the proposed architecture improve training and inference speed while maintaining or improving F1 on benchmark datasets?
• RQ4 What is the impact of dependency and position embeddings on end-to-end RE performance?
• RQ5 Is the model interpretable via visualization of hidden pooling layers and intermediate decisions?

主要发现

• Achieves competitive F1 scores on CoNLL04 and ADE, with notable speedups in training and testing (7–10x faster test time on CoNLL04; 4.5s test time for ADE variants).
• On CoNLL04, Relation-Metric (Ours) attains F1 around 84.57 for RE with 84.46 P and 84.67 R for NER.
• On ADE, Relation-Metric (Ours) achieves RE F1 of 77.29 with NER F1 of 87.11.
• Replicates or surpasses prior state-of-the-art methods while avoiding expensive decoding steps such as beam search.
• The model uses a 3x3 2D convolutional pooling strategy over a 3D input (G, D, P) to integrate local and global features.
• Training and inference times are explicitly reported, illustrating practical efficiency gains.