[Paper Review] Winograd Schema - Knowledge Extraction Using Narrative Chains

Vatsal Mahajan|arXiv (Cornell University)|Jan 8, 2018

Topic Modeling2 references3 citations

TL;DR

This paper proposes a method to extract commonsense knowledge from narrative event chains to improve performance on the Winograd Schema Challenge (WSC), a benchmark for machine reasoning. By modeling events and their participants as Event-Based Conditional Commonsense (ECC) templates, the approach captures causal and temporal dependencies, achieving strong results on a subset of WSC tasks through narrative chain analysis.

ABSTRACT

The Winograd Schema Challenge (WSC) is a test of machine intelligence, designed to be an improvement on the Turing test. A Winograd Schema consists of a sentence and a corresponding question. To successfully answer these questions, one requires the use of commonsense knowledge and reasoning. This work focuses on extracting common sense knowledge which can be used to generate answers for the Winograd schema challenge. Common sense knowledge is extracted based on events (or actions) and their participants; called Event-Based Conditional Commonsense (ECC). I propose an approach using Narrative Event Chains [Chambers et al., 2008] to extract ECC knowledge. These are stored in templates, to be later used for answering the WSC questions. This approach works well with respect to a subset of WSC tasks.

Motivation & Objective

• To address the limitations of current machine reasoning systems in understanding commonsense knowledge required for the Winograd Schema Challenge (WSC).
• To develop a knowledge extraction method that captures causal and temporal dependencies between events and their participants.
• To improve WSC performance by modeling narrative event chains as structured commonsense knowledge templates.
• To demonstrate the effectiveness of narrative chains in encoding real-world reasoning for AI systems.

Proposed method

• The approach uses Narrative Event Chains (NECs) from Chambers et al. (2008) to model sequences of events and their participants.
• Events and their participants are formalized as Event-Based Conditional Commonsense (ECC) templates to encode causal and temporal dependencies.
• ECC templates are constructed from natural language narratives by identifying triggers, participants, and event relations.
• The extracted ECCs are stored as reusable knowledge structures for downstream WSC question answering.
• The method focuses on a subset of WSC examples where narrative chains provide sufficient context for disambiguation.
• The system leverages existing narrative structure to infer implicit commonsense knowledge without requiring external knowledge bases.

Experimental results

Research questions

• RQ1 Can narrative event chains be effectively used to extract commonsense knowledge for the Winograd Schema Challenge?
• RQ2 How well do ECC templates derived from narrative chains perform on WSC tasks compared to other knowledge extraction methods?
• RQ3 What subset of WSC examples can be resolved using narrative chain-based reasoning?
• RQ4 To what extent do event-based conditional commonsense representations capture the necessary causal and temporal reasoning for WSC?
• RQ5 Can narrative structure alone provide sufficient knowledge for disambiguating pronouns in Winograd schemas?

Key findings

• The proposed method achieves strong performance on a subset of Winograd Schema Challenge tasks by leveraging narrative event chains.
• Event-Based Conditional Commonsense (ECC) templates effectively encode commonsense knowledge required for pronoun resolution in WSC.
• Narrative chains provide a structured, interpretable way to model real-world causal and temporal dependencies relevant to WSC.
• The approach demonstrates that narrative structure alone can support reasoning in cases where context and event sequencing are sufficient.
• The method shows promise for scalable commonsense knowledge extraction without reliance on large external knowledge bases.
• The results indicate that narrative chains are a viable source of commonsense knowledge for machine reasoning tasks.