[Paper Review] Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking

Masaru Isonuma, Junichiro Mori+1 more|arXiv (Cornell University)|Jun 13, 2019

Topic Modeling42 references4 citations

TL;DR

This paper proposes StrSum, an unsupervised neural model for abstractive single-document summarization of product reviews by inducing a latent discourse tree without external parsers. By recursively reconstructing parent sentences from their children and using DiscourseRank to prioritize key sentences, the model generates high-quality summaries that abstract the full review, outperforming unsupervised methods and matching or exceeding supervised models on long reviews.

ABSTRACT

This paper focuses on the end-to-end abstractive summarization of a single product review without supervision. We assume that a review can be described as a discourse tree, in which the summary is the root, and the child sentences explain their parent in detail. By recursively estimating a parent from its children, our model learns the latent discourse tree without an external parser and generates a concise summary. We also introduce an architecture that ranks the importance of each sentence on the tree to support summary generation focusing on the main review point. The experimental results demonstrate that our model is competitive with or outperforms other unsupervised approaches. In particular, for relatively long reviews, it achieves a competitive or better performance than supervised models. The induced tree shows that the child sentences provide additional information about their parent, and the generated summary abstracts the entire review.

Motivation & Objective

• To address the high cost of creating reference summaries for supervised summarization in new domains.
• To enable end-to-end abstractive summarization of single product reviews without human-annotated summaries.
• To model reviews as latent discourse trees where child sentences explain their parent, enabling hierarchical abstraction.
• To improve summary quality by ranking sentence importance via discourse structure depth (DiscourseRank).
• To achieve competitive performance with supervised models, especially on longer reviews, without supervision.

Proposed method

• StrSum induces a latent discourse tree end-to-end by recursively reconstructing parent sentences from their children using Bi-GRU encoders and max-pooling for sentence representations.
• The model learns a language model through reconstruction loss, training to generate parent sentences from their child sentences.
• DiscourseRank assigns importance scores to sentences based on the number of descendants in the induced tree, emphasizing central nodes.
• The summary is generated from the root of the induced discourse tree, which represents the overall review point.
• Sentence embeddings are computed via bidirectional GRUs with max-pooling over hidden states to capture contextual representations.
• The architecture avoids external discourse parsers and instead learns discourse structure directly from the data through reconstruction.

Experimental results

Research questions

• RQ1 Can an unsupervised model induce a meaningful latent discourse structure in single product reviews without external parsing?
• RQ2 Does reconstructing parent sentences from their children lead to better abstractive summarization than existing unsupervised methods?
• RQ3 Can discourse structure depth (via DiscourseRank) effectively identify the main review points for summary generation?
• RQ4 How does the model perform on long reviews compared to supervised baselines?
• RQ5 Can the induced discourse tree reveal coherent hierarchical relationships between sentences?

Key findings

• StrSum outperforms other unsupervised models on single-document review summarization, particularly on longer reviews.
• For long reviews, StrSum achieves performance competitive with or better than supervised models, demonstrating strong generalization.
• The induced discourse trees are deeper and more complex than those from supervised baselines like StrAtt, with greater average height (e.g., 3.06 for Toys & Games), indicating richer hierarchical structure.
• DiscourseRank successfully identifies evaluative sentences as high-importance nodes, improving focus on main review points (e.g., highlighting quality issues in goggles review).
• In failure cases, the model misprioritizes aspects (e.g., favoring video quality over poor audio), showing limitations when contrasting aspects have similar tree depth.
• The model generates summaries that abstract the entire review, with child sentences providing explanatory details about their parent, confirming the discourse structure assumption.