[Paper Review] InScript: Narrative texts annotated with script information

Ashutosh Modi, Tatjana Anikina+2 more|arXiv (Cornell University)|Mar 15, 2017

Natural Language Processing Techniques14 references3 citations

TL;DR

This paper introduces InScript, a corpus of 1,000 narrative stories annotated with script knowledge, including event and participant types, and coreference relations. The corpus enables research on script-based language understanding, showing high lexical diversity and strong alignment potential with the DeScript corpus for modeling script knowledge in NLP.

ABSTRACT

This paper presents the InScript corpus (Narrative Texts Instantiating Script structure). InScript is a corpus of 1,000 stories centered around 10 different scenarios. Verbs and noun phrases are annotated with event and participant types, respectively. Additionally, the text is annotated with coreference information. The corpus shows rich lexical variation and will serve as a unique resource for the study of the role of script knowledge in natural language processing.

Motivation & Objective

• To create a large-scale, narrative-based corpus annotated with script knowledge to study how script understanding supports text comprehension.
• To investigate the role of script knowledge in guiding reader expectations, coreference resolution, and common-sense inference.
• To provide a resource that bridges generic script descriptions (DeScript) with instantiated narratives (InScript) for text-to-script alignment research.
• To analyze lexical variation in narrative texts compared to generic script descriptions, using MTLD and entropy metrics.
• To support the development of models for surprisal and information density in written language using script-annotated texts.

Proposed method

• Collected 1,000 stories via Amazon Mechanical Turk, with 100 stories per of 10 distinct scenarios (e.g., taking a bath, baking a cake, flying in an airplane).
• Annotated verbs with event types (e.g., ScrEv_turn_water_on) and noun phrases with participant types (e.g., ScrPart_bather, ScrPart_tap).
• Applied coreference annotation to study dependencies between script structure and pronoun or definite NP resolution.
• Used Measure of Textual Lexical Diversity (MTLD) with a threshold of 0.71 to compare lexical variation between InScript and DeScript.
• Computed entropy over verb lemmas to quantify lexical realization variance for script events, excluding events with fewer than 10 occurrences.
• Aligned InScript narratives with DeScript’s generic event sequences (ESDs) to enable future automatic text-to-script mapping.

Experimental results

Research questions

• RQ1 How does script knowledge influence coreference resolution and contextual integration in narrative texts?
• RQ2 To what extent do narrative texts exhibit lexical variation compared to generic script descriptions?
• RQ3 How do different scenarios (e.g., simple vs. complex) affect lexical diversity and event realization in narrative texts?
• RQ4 Can entropy and MTLD metrics reliably capture differences in lexical realization between narrative and abstract script descriptions?
• RQ5 How well can narrative texts be aligned with generic script structures (e.g., DeScript) for downstream NLP tasks?

Key findings

• InScript contains 1,000 narrative stories with an average of 200 words per story, totaling approximately 200,000 words, and is annotated with event types, participant types, and coreference.
• The corpus exhibits significantly higher lexical diversity than DeScript, as measured by MTLD, with InScript showing consistently higher values across all 10 scenarios.
• Entropy analysis revealed that event types like 'spend_time_train' have high lexical variance (high entropy), while others like 'wait' show low variance, indicating diverse linguistic realizations for complex activities.
• The 'flying in an airplane' scenario showed the highest lexical diversity in both InScript and DeScript, suggesting that complex scenarios naturally lead to greater lexical variation.
• The average number of mentions for the most frequent participant types (e.g., 'bather', 'bathtub') was significantly higher than for less frequent types, indicating a coreference bias toward canonical script participants.
• The corpus supports the hypothesis that narrative texts instantiate scripts with rich lexical variation, making them ideal for studying script knowledge in context-aware NLP systems.