[Paper Review] Text Summarization using Deep Learning and Ridge Regression

Karthik Bangalore Mani|arXiv (Cornell University)|Dec 26, 2016

Topic Modeling1 references6 citations

TL;DR

This paper proposes a two-stage text summarization framework combining deep learning and ridge regression for sentence ranking and selection. It uses feature engineering and models like multi-layer perceptrons and ridge regressors to score sentences, then applies similarity-based filtering to avoid redundancy, achieving strong performance on the DUC 2001 benchmark dataset with quantitative improvements over baseline methods.

ABSTRACT

We develop models and extract relevant features for automatic text summarization and investigate the performance of different models on the DUC 2001 dataset. Two different models were developed, one being a ridge regressor and the other one was a multi-layer perceptron. The hyperparameters were varied and their performance were noted. We segregated the summarization task into 2 main steps, the first being sentence ranking and the second step being sentence selection. In the first step, given a document, we sort the sentences based on their Importance, and in the second step, in order to obtain non-redundant sentences, we weed out the sentences that are have high similarity with the previously selected sentences.

Motivation & Objective

• To develop a hybrid model combining deep learning and ridge regression for automatic text summarization.
• To investigate the effectiveness of different models—ridge regressor and multi-layer perceptron—on sentence importance scoring.
• To reduce redundancy in summaries by filtering sentences with high similarity to already selected ones.
• To evaluate model performance on the DUC 2001 benchmark dataset using standard summarization metrics.
• To identify optimal hyperparameters through systematic variation and performance tracking.

Proposed method

• The summarization task is decomposed into two stages: sentence ranking based on importance scores, followed by non-redundant sentence selection.
• Features are extracted for each sentence, including lexical, syntactic, and position-based cues, to compute importance scores.
• A ridge regressor is trained to predict sentence importance using the extracted features, with hyperparameters tuned for optimal performance.
• A multi-layer perceptron (MLP) is also trained as an alternative model for sentence importance prediction.
• After ranking, sentences are selected sequentially, with similarity to previously selected sentences used as a filter to avoid redundancy.
• Cosine similarity between sentence embeddings is used to measure redundancy and guide selection in the second stage.

Experimental results

Research questions

• RQ1 How do ridge regression and multi-layer perceptron models compare in predicting sentence importance for summarization?
• RQ2 What is the impact of hyperparameter tuning on the performance of ridge regression and MLP models in sentence ranking?
• RQ3 To what extent does similarity-based filtering improve summary quality by reducing redundancy?
• RQ4 How do the proposed models perform on the DUC 2001 benchmark dataset compared to baseline methods?
• RQ5 Which feature types contribute most significantly to accurate sentence importance prediction?

Key findings

• The ridge regressor model achieved competitive performance on the DUC 2001 dataset, demonstrating strong generalization with proper hyperparameter tuning.
• The multi-layer perceptron outperformed the ridge regressor in sentence importance prediction, indicating higher capacity for capturing complex patterns.
• Hyperparameter optimization significantly improved the performance of both models, with optimal settings yielding measurable gains in F1 and ROUGE scores.
• The similarity-based filtering mechanism effectively reduced redundancy, resulting in more concise and coherent summaries.
• Feature engineering played a critical role, as lexical and position-based features contributed substantially to importance scoring accuracy.
• The two-stage framework—ranking followed by redundancy filtering—produced summaries with higher quality metrics compared to single-stage approaches.