[Paper Review] Creating Suspenseful Stories: Iterative Planning with Large Language Models

Kaige Xie, Mark Riedl|arXiv (Cornell University)|Feb 27, 2024

AI-based Problem Solving and Planning4 citations

TL;DR

This paper proposes a theory-grounded, iterative-prompting method for generating suspenseful stories using large language models (LLMs) without supervised data. By iteratively planning protagonist actions and adversarially undermining them, the method increases suspense, achieving significantly higher human-rated suspense than baselines, marking the first zero-shot LLM-based approach to suspenseful story generation.

ABSTRACT

Automated story generation has been one of the long-standing challenges in NLP. Among all dimensions of stories, suspense is very common in human-written stories but relatively under-explored in AI-generated stories. While recent advances in large language models (LLMs) have greatly promoted language generation in general, state-of-the-art LLMs are still unreliable when it comes to suspenseful story generation. We propose a novel iterative-prompting-based planning method that is grounded in two theoretical foundations of story suspense from cognitive psychology and narratology. This theory-grounded method works in a fully zero-shot manner and does not rely on any supervised story corpora. To the best of our knowledge, this paper is the first attempt at suspenseful story generation with LLMs. Extensive human evaluations of the generated suspenseful stories demonstrate the effectiveness of our method.

Motivation & Objective

• To address the under-explored challenge of generating suspenseful stories with large language models (LLMs), which are typically ineffective at capturing suspense despite advances in fluency and length.
• To develop a method that does not rely on supervised story corpora, enabling fully zero-shot suspenseful story generation grounded in psychological theory.
• To investigate the factors influencing reader suspense perception, including clue setup, information timing, and empathy for the protagonist.
• To validate the effectiveness of the proposed method through extensive human evaluations and controlled intervention studies.

Proposed method

• The method is grounded in a psychological theory of suspense: readers feel suspense when the number or quality of ways to avoid a negative outcome diminishes.
• It uses iterative prompting to generate a protagonist’s potential actions toward a goal, then adversarially constructs story conditions that cause each plan to fail.
• This process is repeated multiple times to build a suspenseful story outline before expanding into full narrative text.
• The approach operates in a fully zero-shot manner, requiring no fine-tuning or supervised training data.
• The method incorporates theory-based design choices such as clue placement and timing of failure revelations to influence suspense perception.
• Human evaluations are conducted using controlled surveys to assess suspense levels, empathy, and preference across different story variants.

Experimental results

Research questions

• RQ1 Can a theory-grounded, zero-shot prompting method effectively generate suspenseful stories using LLMs without any supervised data?
• RQ2 How do clue setup and the timing of failure information revelation affect reader perceptions of suspense?
• RQ3 To what extent does reader empathy for the protagonist influence the perceived level of suspense?
• RQ4 How does the proposed iterative-prompting method compare to baseline prompting methods in generating suspenseful narratives?

Key findings

• 57.9% of participants reported higher suspense when reading stories with clue setups indicating impending failure, compared to 10.9% who preferred stories without clues, with results significant at p < 0.05.
• 42.1% of participants felt more suspense when failure conditions were revealed beforehand, compared to 36.3% when revealed afterward, though this difference was not statistically significant.
• A strong positive correlation was observed between reader empathy and perceived suspense, with high suspense levels typically associated with high empathy, though some participants reported low suspense even with high empathy.
• The proposed method significantly outperformed baseline prompting methods in human evaluations, demonstrating its effectiveness in generating suspenseful narratives.
• The study confirms that clue-based setup of failure scenarios is a key factor in increasing suspense perception, supporting the theoretical foundation of the method.
• The method enables fully zero-shot suspenseful story generation that is both effective and grounded in cognitive psychology, representing a novel contribution to LLM-based storytelling.