[Paper Review] A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution

Tuan Lai, Heng Ji+4 more|arXiv (Cornell University)|Apr 4, 2021

Topic Modeling38 references4 citations

TL;DR

This paper proposes a context-dependent gated module (CDGM) that adaptively integrates noisy symbolic features—such as event modality, polarity, and tense—into event coreference resolution using SpanBERT. By dynamically weighting symbolic features based on context and applying noise regularization during training, the method achieves state-of-the-art F1 scores of 59.76 on ACE 2005 and 58.41 on KBP 2016, significantly outperforming baselines and simple concatenation strategies.

ABSTRACT

Event coreference resolution is an important research problem with many applications. Despite the recent remarkable success of pretrained language models, we argue that it is still highly beneficial to utilize symbolic features for the task. However, as the input for coreference resolution typically comes from upstream components in the information extraction pipeline, the automatically extracted symbolic features can be noisy and contain errors. Also, depending on the specific context, some features can be more informative than others. Motivated by these observations, we propose a novel context-dependent gated module to adaptively control the information flows from the input symbolic features. Combined with a simple noisy training method, our best models achieve state-of-the-art results on two datasets: ACE 2005 and KBP 2016.

Motivation & Objective

• To address the challenge of incorporating noisy symbolic features—such as event modality, polarity, and tense—into event coreference resolution, which are often error-prone when automatically predicted.
• To overcome the limitations of fixed feature integration methods like concatenation, which treat all features equally regardless of context or reliability.
• To develop a generalizable framework that can selectively utilize informative symbolic features based on contextual cues, improving robustness and performance.
• To demonstrate that combining context-aware feature gating with noise regularization enhances model generalization, especially on low-accuracy symbolic features.

Proposed method

• A context-dependent gated module (CDGM) is introduced to dynamically control the flow of information from symbolic features based on the surrounding context, using a learnable gating mechanism.
• Symbolic features (e.g., modality, polarity) are embedded into learnable vectors and processed through the CDGM, which computes attention-like weights to modulate their contribution to the final representation.
• The CDGM computes a context-aware weighted sum of symbolic feature representations, where the weights are computed using a feed-forward network conditioned on the mention's contextual representation.
• A simple noise regularization technique is applied during training by randomly corrupting symbolic features, improving robustness to prediction errors.
• The model uses SpanBERT as the contextual encoder and combines mention representations with symbolic features via the CDGM before computing pairwise similarity for coreference prediction.
• The end-to-end model is trained jointly on coreference resolution and symbolic feature prediction, though the primary focus is on improving coreference performance through better symbolic feature integration.

Experimental results

Research questions

• RQ1 Can a context-dependent gating mechanism improve event coreference resolution by selectively utilizing symbolic features based on contextual relevance?
• RQ2 How does the performance of symbolic features vary across different types (e.g., modality vs. subtype), and can a dynamic module mitigate the impact of noisy features?
• RQ3 Does noise regularization during training enhance model robustness when symbolic features are imperfectly predicted?
• RQ4 Can a unified framework effectively incorporate diverse symbolic features (e.g., modality, polarity, tense) without requiring task-specific architectures?
• RQ5 How does the proposed method compare to simple concatenation or multi-task learning in terms of coreference resolution accuracy?

Key findings

• The proposed CDGM with noise regularization achieves a state-of-the-art F1 score of 59.76 on the ACE 2005 dataset, significantly outperforming the baseline SpanBERT model.
• The method achieves a 3.73-point improvement on the modality feature and 3.12-point improvement on genericity, indicating strong gains for noisy symbolic features.
• The CDGM outperforms simple concatenation across all symbolic feature types, with the largest gains observed on features with lower prediction accuracy (e.g., modality, genericity).
• The model achieves an F1 score of 58.41 on KBP 2016, demonstrating strong generalization across different datasets and annotation guidelines.
• Multi-task learning with joint feature prediction and coreference resolution yields only 56.5 F1 on ACE 2005, underscoring the superiority of the proposed gated integration over joint training.
• Qualitative examples show that the model correctly resolves coreference by leveraging symbolic features like modality and polarity, which distinguish events that are semantically similar but contextually distinct.