[Paper Review] Personalized Academic Research Paper Recommendation System

Joonseok Lee, Kisung Lee+1 more|arXiv (Cornell University)|Apr 19, 2013

Recommender Systems and Techniques19 references27 citations

TL;DR

This paper proposes a personalized academic research paper recommendation system that uses web crawling, text-based similarity via a bag-of-words model, and collaborative filtering to recommend relevant papers. Evaluation with real researchers showed high relevance (average 3.88/6) and strong user satisfaction, demonstrating the system's effectiveness in reducing manual search effort.

ABSTRACT

A huge number of academic papers are coming out from a lot of conferences and journals these days. In these circumstances, most researchers rely on key-based search or browsing through proceedings of top conferences and journals to find their related work. To ease this difficulty, we propose a Personalized Academic Research Paper Recommendation System, which recommends related articles, for each researcher, that may be interesting to her/him. In this paper, we first introduce our web crawler to retrieve research papers from the web. Then, we define similarity between two research papers based on the text similarity between them. Finally, we propose our recommender system developed using collaborative filtering methods. Our evaluation results demonstrate that our system recommends good quality research papers.

Motivation & Objective

• To reduce the time and effort researchers spend manually searching for relevant academic papers across conferences and journals.
• To develop a recommendation system that personalizes paper suggestions based on researchers' past work and interests.
• To improve the accuracy and relevance of paper discovery beyond keyword-based searches like Google Scholar or Citeseer.
• To evaluate the system's performance using real user studies with academic researchers.

Proposed method

• A web crawler was implemented to harvest research papers from IEEE Xplore and ACM Digital Library using URL pattern matching and regular expressions.
• Text data from papers were preprocessed using a bag-of-words model, representing documents as term frequency vectors.
• Paper similarity was computed based on textual content using a cosine-like similarity measure on term vectors.
• A collaborative filtering approach, specifically a lazy learning method akin to k-Nearest Neighbors, was used to predict user preferences and recommend top papers.
• The system employs a graphical user interface where users input their name and desired number of recommendations.
• Clustering and neighbor-based recommendation algorithms were applied to group similar papers and generate personalized suggestions.

Experimental results

Research questions

• RQ1 Can a content-based and collaborative filtering approach effectively recommend relevant academic papers to researchers?
• RQ2 How does the system perform in terms of relevance and user satisfaction compared to manual search methods?
• RQ3 To what extent can text similarity and collaborative filtering improve the accuracy of paper recommendations?
• RQ4 How do researchers perceive the usefulness and usability of the system in real-world scenarios?

Key findings

• The system achieved an average relevance score of 3.88 out of 6.0 from user evaluations, indicating strong perceived relevance.
• Subjects reported reading only about two out of ten recommended papers, highlighting the system's ability to filter high-quality, relevant results.
• All participants rated the system 6.0 out of 6.0 in usability and usefulness, indicating high user satisfaction.
• Subjects expressed particular interest in papers authored by peers or collaborators, suggesting social context enhances recommendation appeal.
• The system demonstrated feasibility and effectiveness despite using only content-based features and no explicit user profile beyond past work.
• Limitations were identified, including difficulty distinguishing nuanced topics and handling users with shifting research interests.