[Paper Review] New Alignment Methods for Discriminative Book Summarization

David Bamman, Noah A. Smith|arXiv (Cornell University)|May 6, 2013

Natural Language Processing Techniques17 references16 citations

TL;DR

This paper proposes two novel hidden Markov model (HMM)-based alignment methods for unsupervised discriminative book summarization, addressing the extreme length disparity between full books and summaries. The passage model aligns document passages to summary sentences using unigram language model probabilities, while the token model extends HMM alignment to long, sparsely aligned documents; both methods improve extractive book summarization performance on a Project Gutenberg-Wikipedia dataset with a 1.2% summary-to-book ratio.

ABSTRACT

We consider the unsupervised alignment of the full text of a book with a human-written summary. This presents challenges not seen in other text alignment problems, including a disparity in length and, consequent to this, a violation of the expectation that individual words and phrases should align, since large passages and chapters can be distilled into a single summary phrase. We present two new methods, based on hidden Markov models, specifically targeted to this problem, and demonstrate gains on an extractive book summarization task. While there is still much room for improvement, unsupervised alignment holds intrinsic value in offering insight into what features of a book are deemed worthy of summarization.

Motivation & Objective

• To address the challenge of unsupervised alignment between long-form books and short human-written summaries, where traditional word-level alignment fails due to extreme length disparity.
• To develop alignment techniques that recognize that entire passages or chapters may map to single summary sentences, rather than assuming one-to-one word or phrase correspondence.
• To adapt discriminative summarization methods—typically trained on news articles—to the book summarization domain, where summary-to-source ratios are drastically lower (1.2% on average).
• To explore the intrinsic value of unsupervised alignment in revealing which textual features are deemed most salient for summarization.
• To demonstrate that HMM-based alignment models can effectively capture long-range, sparse alignments in book-text summarization tasks.

Proposed method

• The passage model defines HMM states as contiguous passages in the source book, with observations being summary sentences; emission probabilities are derived from unigram language model likelihoods of the summary sentence under the passage.
• The token model treats individual source tokens as HMM states and summary tokens as observations, using lexical identity and synonymy for emission modeling, with transitions defined by distance bins to model sparse alignment.
• Both models use HMMs with state transition and emission probabilities learned via expectation-maximization, with sequence length conditioned explicitly to avoid end-state modeling.
• The passage model uses a generative approach where the probability of a summary sentence given a passage is computed via the HMM likelihood, enabling alignment of long, coherent segments to summary content.
• The token model generalizes the HMM alignment framework to handle long documents by restricting transitions to local distance bins, improving scalability and alignment accuracy.
• Inference in both models uses the forward-backward algorithm to compute alignment probabilities, and parameter learning is performed via EM to maximize the likelihood of observed summary-sentence alignments.

Experimental results

Research questions

• RQ1 Can HMM-based models effectively align long, low-ratio source documents (e.g., books) to short summaries, where traditional word-level alignment fails?
• RQ2 How can alignment models be adapted to handle the fact that entire passages or chapters may correspond to single summary sentences, rather than individual words or phrases?
• RQ3 To what extent do unigram language model probabilities of summary sentences under source passages improve alignment quality in book summarization?
• RQ4 Can a generalized HMM framework for sparse, long-document alignment outperform standard alignment techniques in extractive book summarization tasks?
• RQ5 What insights into summarization-relevant features can be gained from unsupervised alignment of books and summaries?

Key findings

• The passage model achieves significant gains in extractive book summarization performance by aligning coherent source passages to summary sentences using unigram language model probabilities.
• The token model, which extends HMM alignment to long, sparsely aligned documents via distance-binned transitions, demonstrates improved alignment accuracy over baseline methods.
• On average, summaries are only 1.2% the length of their corresponding books, highlighting the extreme length disparity that challenges standard alignment assumptions.
• The study shows that unsupervised alignment can reveal salient features of a book that are deemed worthy of summarization, offering interpretability beyond performance metrics.
• Both proposed HMM-based methods outperform existing alignment techniques on the extractive book summarization task, particularly in capturing long-range, coarse-grained alignments.
• The results suggest that modeling alignment at the passage level, rather than word level, is essential for effective book summarization due to the disproportionate size of source texts.