[Paper Review] Automatic text summarization: What has been done and what has to be done

Abdelkrime Aries, Djamel Eddine Zegour+1 more|arXiv (Cornell University)|Apr 1, 2019

Topic ModelingComputer Science105 references18 citations

TL;DR

This paper provides a comprehensive review of automatic text summarization (ATS), categorizing existing methods by input type, purpose, and output form, while identifying persistent challenges such as lack of evaluation standards, limited corpora, and insufficient NLP tools. It highlights the need for improved evaluation frameworks and resource availability to advance both extractive and abstractive summarization systems.

ABSTRACT

Summaries are important when it comes to process huge amounts of information. Their most important benefit is saving time, which we do not have much nowadays. Therefore, a summary must be short, representative and readable. Generating summaries automatically can be beneficial for humans, since it can save time and help selecting relevant documents. Automatic summarization and, in particular, Automatic text summarization (ATS) is not a new research field; It was known since the 50s. Since then, researchers have been active to find the perfect summarization method. In this article, we will discuss different works in automatic summarization, especially the recent ones. We will present some problems and limits which prevent works to move forward. Most of these challenges are much more related to the nature of processed languages. These challenges are interesting for academics and developers, as a path to follow in this field.

Motivation & Objective

• To survey and classify existing automatic text summarization (ATS) methods based on input document criteria, purpose, and output form.
• To identify key challenges hindering progress in ATS, including evaluation difficulties, lack of annotated corpora, and insufficient NLP resources.
• To highlight the gap between current evaluation practices and the need for more robust, human-like assessment of summary quality.
• To emphasize the importance of domain-specific and multilingual support in advancing ATS systems.
• To guide future research by outlining unresolved issues such as coherence, readability, and automatic quality assessment in summarization.

Proposed method

• Classifies summarization systems using three criteria: input document source size (mono- vs. multi-document), specificity (domain-specific vs. general-purpose), and form (structured, unstructured, multimedia, etc.).
• Reviews major ATS approaches, including extractive methods (e.g., TF-IDF, centroid-based), abstractive methods, and hybrid techniques using linguistic and statistical features.
• Analyzes the use of machine learning and semi-supervised learning to address data scarcity, especially in low-resource domains.
• Examines evaluation frameworks, including reference-based automatic metrics (e.g., ROUGE) and human evaluation, and their limitations in assessing coherence and readability.
• Discusses the role of domain-specific resources (e.g., biomedical concepts, cue phrases) in improving relevance and reducing ambiguity.
• Proposes that future systems should integrate deeper linguistic analysis and automatic quality control to improve summary fluency and structure.

Experimental results

Research questions

• RQ1 What are the main classifications of automatic text summarization systems based on input, purpose, and output?
• RQ2 Why do current evaluation methods fail to adequately assess the quality of abstractive summaries?
• RQ3 What are the primary challenges in creating reliable and diverse training corpora for automatic text summarization?
• RQ4 How do language-specific NLP tool limitations affect the development of multilingual summarization systems?
• RQ5 What are the key unresolved issues in evaluating summary readability and coherence, and how can they be addressed?

Key findings

• Mono-document and multi-document summarization remain the dominant input classifications, with multi-document systems facing challenges due to redundancy across sources.
• Domain-specific summarization systems, such as those using biomedical concepts or juridical cue phrases, show improved relevance detection compared to general-purpose models.
• The lack of standardized, high-quality corpora for training and evaluation remains a major bottleneck, especially for low-resource languages and non-news genres.
• Automatic evaluation metrics like ROUGE are more effective for extractive summarization than abstractive methods, as they often fail to capture semantic and syntactic quality.
• Human evaluation is costly and inconsistent due to subjective variation, yet remains the gold standard for assessing informativeness and readability.
• Readability and coherence are under-evaluated aspects in current ATS research, despite being critical for user comprehension and system usability.