[Paper Review] Event Linking: Grounding Event Mentions to Wikipedia

Xiaodong Yu, Wenpeng Yin+2 more|arXiv (Cornell University)|Dec 15, 2021

Wikis in Education and Collaboration4 citations

TL;DR

This paper introduces Event Linking, a novel NLP task that grounds event mentions in text to the most relevant Wikipedia pages for enhanced contextual understanding. It proposes EveLink, an event-specific linking model that incorporates local entities and types, achieving state-of-the-art performance on in-domain and out-of-domain benchmarks, with significant gains over SOTA entity linking models, especially on seen events and easy cases.

ABSTRACT

Comprehending an article requires understanding its constituent events. However, the context where an event is mentioned often lacks the details of this event. A question arises: how can the reader obtain more knowledge about this particular event in addition to what is provided by the local context in the article? This work defines Event Linking, a new natural language understanding task at the event level. Event linking tries to link an event mention appearing in an article to the most appropriate Wikipedia page. This page is expected to provide rich knowledge about what the event mention refers to. To standardize the research in this new direction, we contribute in four-fold. First, this is the first work in the community that formally defines Event Linking task. Second, we collect a dataset for this new task. Specifically, we automatically gather training set from Wikipedia, and then create two evaluation sets: one from the Wikipedia domain, reporting the in-domain performance, and a second from the real-world news domain, to evaluate out-of-domain performance. Third, we retrain and evaluate two state-of-the-art (SOTA) entity linking models, showing the challenges of event linking, and we propose an event-specific linking system EVELINK to set a competitive result for the new task. Fourth, we conduct a detailed and insightful analysis to help understand the task and the limitation of the current model. Overall, as our analysis shows, Event Linking is a considerably challenging and essential task requiring more effort from the community. Data and code are available here: https://github.com/CogComp/event-linking.

Motivation & Objective

• To formalize Event Linking as a new natural language understanding task, distinct from entity linking, to improve contextual comprehension through event-specific knowledge grounding.
• To collect and release a high-quality, balanced dataset for event linking, including in-domain (Wikipedia) and out-of-domain (New York Times) test sets with hard and easy cases.
• To evaluate the limitations of existing SOTA entity linking models on event linking and demonstrate the need for task-specific modeling.
• To design and implement EveLink, a novel event linking model that enhances performance by incorporating event arguments (entities) and entity types.
• To conduct in-depth error analysis to identify key challenges such as temporal ambiguity, unrelated context, and sub-event detection, guiding future research.

Proposed method

• Event Linking is defined as linking an event mention (e.g., a verb or nominal phrase) in a text to the most relevant Wikipedia page that provides detailed knowledge about the event.
• Training data is automatically collected from Wikipedia hyperlinks, while test sets are created from Wikipedia (in-domain) and 20-year New York Times articles (out-of-domain), with "Nil" labels for events not in Wikipedia.
• EveLink models event mentions by encoding the event trigger and its arguments (entities) from the local context, using a dual-encoder architecture to compute similarity with Wikipedia titles.
• The model incorporates entity types as additional features, improving representation learning and performance on ambiguous or complex events.
• A confidence threshold of 50% is used to predict "Nil" for low-confidence predictions, though the paper acknowledges this is a heuristic and not optimal.
• The model is fine-tuned on the training set and evaluated using accuracy, with ablation studies to analyze the contribution of entities and types.

Experimental results

Research questions

• RQ1 How does event linking differ from entity linking in terms of structure, context dependency, and knowledge requirements?
• RQ2 What are the key challenges in grounding events to Wikipedia, especially in real-world news where events may not be explicitly documented?
• RQ3 How do existing SOTA entity linking models perform on event linking, and what improvements are needed to address event-specific complexities?
• RQ4 To what extent do local entities and their types improve event linking performance, and how do they mitigate ambiguity?
• RQ5 What are the dominant error patterns in event linking, and what do they reveal about the limitations of current models and the need for document-level or temporal reasoning?

Key findings

• EveLink outperforms two SOTA entity linking models, BLINK and GENRE, on the event linking benchmark, achieving a 10.14-point accuracy gain on the Wikipedia test set and a 2.73-point gain on the New York Times test set.
• The inclusion of local entities in the event representation boosts performance by 10.14 accuracy points on the in-domain test set and 2.73 points on the out-of-domain set.
• Adding entity types further improves performance by 1.48 points on the Wikipedia set and 4.03 points on the NYT set, indicating that type information is particularly valuable for out-of-domain generalization.
• A major error pattern involves repeating events such as awards or sports games, where models fail to infer the correct year when not explicitly stated, highlighting the need for better temporal reasoning.
• Many errors arise from unrelated local context—e.g., football-related entities misleading models into linking to sports events instead of historical events like World War II—indicating that context alone is insufficient for accurate linking.
• Sub-events and events only mentioned in subsections of Wikipedia pages pose significant challenges, as the model cannot access full document context, suggesting a need for document-level modeling in future work.