[Paper Review] Topic Extraction and Bundling of Related Scientific Articles

Shameem Ahamad Puthiva Parambath|arXiv (Cornell University)|Dec 21, 2012

Advanced Text Analysis Techniques41 references3 citations

TL;DR

This paper proposes a two-step method for automatically extracting topics and bundling related scientific articles using Latent Dirichlet Allocation (LDA) for topic modeling and hierarchical agglomerative clustering with combined content and co-authorship similarity. The approach achieves 89.1% agreement in user studies and outperforms content-only clustering in 82.7% of comparative evaluations, demonstrating improved bundling semantics for digital libraries and information retrieval systems.

ABSTRACT

Automatic classification of scientific articles based on common characteristics is an interesting problem with many applications in digital library and information retrieval systems. Properly organized articles can be useful for automatic generation of taxonomies in scientific writings, textual summarization, efficient information retrieval etc. Generating article bundles from a large number of input articles, based on the associated features of the articles is tedious and computationally expensive task. In this report we propose an automatic two-step approach for topic extraction and bundling of related articles from a set of scientific articles in real-time. For topic extraction, we make use of Latent Dirichlet Allocation (LDA) topic modeling techniques and for bundling, we make use of hierarchical agglomerative clustering techniques. We run experiments to validate our bundling semantics and compare it with existing models in use. We make use of an online crowdsourcing marketplace provided by Amazon called Amazon Mechanical Turk to carry out experiments. We explain our experimental setup and empirical results in detail and show that our method is advantageous over existing ones.

Motivation & Objective

• To address the challenge of grouping large volumes of scientific articles into coherent, meaningful bundles for improved information retrieval and digital library organization.
• To develop a scalable, real-time bundling system that integrates latent topics and authorship relationships for enhanced semantic coherence.
• To validate the effectiveness of hybrid similarity (content + co-authorship) over content-only similarity in article bundling through empirical user studies.
• To provide a generalized, extensible framework for article bundling applicable to unstructured and structured scientific data.

Proposed method

• Utilizes Latent Dirichlet Allocation (LDA) to extract latent topics from scientific articles, modeling each document as a mixture of topics.
• Applies hierarchical agglomerative clustering to group articles based on inter-textual distance and co-authorship similarity, combining content and authorship features.
• Employs Gibbs sampling via the MALLET toolkit to infer topic distributions and word-topic assignments in the LDA model.
• Integrates document-level topic distributions with authorship co-occurrence to compute a composite similarity measure for clustering.
• Uses Amazon Mechanical Turk to conduct independent and comparative user studies for evaluating bundling quality and semantic coherence.
• Employs survey-based evaluation to assess bundle similarity and preference between content-only and extended similarity-based clustering.

Experimental results

Research questions

• RQ1 Can a two-step approach combining LDA topic modeling and hybrid similarity clustering effectively bundle related scientific articles?
• RQ2 How does the inclusion of co-authorship similarity improve the semantic coherence of article bundles compared to content-only similarity?
• RQ3 To what extent do human evaluators perceive the proposed bundling method as more coherent than existing content-based approaches?
• RQ4 What is the relative performance of the proposed method in terms of user agreement and preference across diverse scientific topics?

Key findings

• The proposed method achieved a 100% agreement rate in the independent study for the Information Retrieval topic, indicating strong semantic coherence in article bundles.
• In the Machine Learning topic, 84.2% of workers affirmed that articles within the same bundle were similar, supporting the method’s effectiveness in diverse domains.
• The comparative study showed that 82.7% of users preferred bundles generated using extended similarity (content + co-authorship) over content-only bundles.
• Overall, 89.1% of users in the independent study confirmed that articles in the same bundle were very similar, indicating high semantic quality of the bundling process.
• The method demonstrated consistent superiority across topics, with preference ratios ranging from 80% to 86% in comparative evaluations.
• The results suggest that incorporating co-authorship and topic similarity significantly enhances the interpretability and coherence of article bundles.