[Paper Review] Generating Music from Literature

Hannah Davis, Saif M. Mohammad|arXiv (Cornell University)|Mar 10, 2014

Music and Audio Processing37 references4 citations

TL;DR

This paper introduces TransProse, a system that generates musical compositions from literary texts by mapping emotional content—specifically emotion word density and valence—to musical elements such as tempo, key, pitch, and note duration. The method produces coherent, emotionally resonant piano pieces that reflect the narrative mood and emotional dynamics of novels, with evaluations showing that pieces based on top emotions and overall emotional density yield more musically cohesive results.

ABSTRACT

We present a system, TransProse, that automatically generates musical pieces from text. TransProse uses known relations between elements of music such as tempo and scale, and the emotions they evoke. Further, it uses a novel mechanism to determine sequences of notes that capture the emotional activity in the text. The work has applications in information visualization, in creating audio-visual e-books, and in developing music apps.

Motivation & Objective

• To develop a system that automatically generates music from literary texts based on emotional content.
• To map emotional dynamics in novels to musical parameters such as tempo, pitch, and note duration.
• To create audio-visual e-books and music apps that enhance reading with mood-aligned music.
• To explore the feasibility of using emotion word density as a basis for musical composition.
• To evaluate the perceptual quality and emotional fidelity of generated music through comparative analysis.

Proposed method

• TransProse uses emotion word density in text to determine overall musical tempo, with higher densities leading to faster tempos (e.g., >170 BPM for active novels like Peter Pan).
• The system assigns musical key and pitch range based on the dominant emotion: positive emotions raise the pitch by one octave, negative emotions lower it by one octave.
• Note sequences are generated by tracking changes in emotion word density over text segments, with higher density resulting in more frequent and shorter notes.
• The system applies constraints to ensure musical coherence, such as limiting note transitions and using predefined rhythmic patterns to avoid cacophony.
• Multiple melodies are generated based on the top two emotions and overall emotional density, with each melody assigned to a separate voice or staff.
• The system uses a modular structure with sections and sub-sections to reflect narrative structure, allowing for dynamic variation in musical activity.

Experimental results

Research questions

• RQ1 Can emotional content in literature be systematically mapped to musical parameters such as tempo, key, and note sequence to generate coherent music?
• RQ2 How does the distribution of emotion word density in a novel affect the perceived mood and structure of the generated music?
• RQ3 What impact do multiple emotional melodies (based on top emotions) have on the perceived musical quality and emotional fidelity?
• RQ4 How do variations in sectioning and note duration affect the perception of emotional activity in the music?
• RQ5 To what extent does the system produce music that is both musically coherent and emotionally aligned with the source text?

Key findings

• Novels with high happiness or sadness word densities, such as Peter Pan and The Road, generated music an octave higher or lower than average, respectively, resulting in lighter or darker tonal qualities.
• Fast-paced novels like A Clockwork Orange and Lord of the Flies produced tempos exceeding 170 beats per minute, reflecting high emotional activity, while slower novels like The Road had tempos as low as 42 BPM.
• Pieces based on two melodies (overall density and top emotion) were perceived as more cohesive than those with four melodies, which often sounded chaotic or less unified.
• The final section of A Clockwork Orange showed a dramatic shift from chaotic to structured music, mirroring the protagonist’s psychological transformation in the novel.
• In Heart of Darkness, increasing note density and decreasing note duration over time reflected the rising tension in the narrative, with music becoming more active as the story progressed.
• Human evaluation of alternative configurations revealed that using C minor for relatively positive novels reduced emotional coherence, suggesting that key choice must align with emotional valence.