[Paper Review] Contextual Document Similarity for Content-based Literature Recommender Systems

Malte Ostendorff|arXiv (Cornell University)|Aug 1, 2020

Topic Modeling4 citations

TL;DR

This paper proposes contextual document similarity for content-based literature recommender systems, modeling similarity as a triple of two documents and a specific context (e.g., methodology, findings). By integrating semantic features from text and citation links with neural embeddings, the approach enables fine-grained, context-aware recommendations, significantly improving retrieval of distantly related yet relevant scientific papers compared to traditional single-score methods.

ABSTRACT

To cope with the ever-growing information overload, an increasing number of digital libraries employ content-based recommender systems. These systems traditionally recommend related documents with the help of similarity measures. However, current document similarity measures simply distinguish between similar and dissimilar documents. This simplification is especially crucial for extensive documents, which cover various facets of a topic and are often found in digital libraries. Still, these similarity measures neglect to what facet the similarity relates. Therefore, the context of the similarity remains ill-defined. In this doctoral thesis, we explore contextual document similarity measures, i.e., methods that determine document similarity as a triple of two documents and the context of their similarity. The context is here a further specification of the similarity. For example, in the scientific domain, research papers can be similar with respect to their background, methodology, or findings. The measurement of similarity in regards to one or more given contexts will enhance recommender systems. Namely, users will be able to explore document collections by formulating queries in terms of documents and their contextual similarities. Thus, our research objective is the development and evaluation of a recommender system based on contextual similarity. The underlying techniques will apply established similarity measures and as well as neural approaches while utilizing semantic features obtained from links between documents and their text.

Motivation & Objective

• To address the limitation of traditional similarity measures that treat documents as monolithic entities, ignoring multiple facets of content.
• To enable expert users to explore literature collections through context-specific queries, such as finding papers with similar methods but different results.
• To develop a recommender system that leverages semantic features from text and citation networks to compute similarity in specific contexts.
• To evaluate the effectiveness of contextual similarity in retrieving relevant, distantly related scientific papers that traditional systems might miss.
• To demonstrate that context-aware similarity improves recommendation quality by capturing nuanced relationships between research papers.

Proposed method

• The method models document similarity as a triple: (seed document, target document, context), where context specifies the aspect of similarity (e.g., methodology, background).
• It combines neural word embeddings (e.g., BERT, GloVe) with citation-based semantic signals to enrich document representations.
• Citation links are used to infer contextual relationships between documents, enhancing semantic understanding beyond raw text.
• The approach applies established similarity measures (e.g., cosine similarity) within each context, producing context-specific similarity scores.
• A multi-context retrieval framework selects recommendations based on similarity in user-specified contexts, enabling analogical search.
• The system integrates text features, citation networks, and pre-trained language models to generate context-aware document embeddings.

Experimental results

Research questions

• RQ1 Can contextual document similarity improve the retrieval of distantly related scientific papers that share only a specific aspect, such as methodology?
• RQ2 How does context-aware similarity compare to traditional single-score similarity in recommending relevant literature?
• RQ3 To what extent can citation networks and text embeddings jointly enhance context-specific similarity modeling?
• RQ4 Can users effectively explore literature collections using queries based on specific similarity contexts?
• RQ5 What is the impact of different context types (e.g., methodology, findings) on recommendation quality and diversity?

Key findings

• The proposed contextual similarity approach significantly improves the retrieval of relevant, non-obvious connections between scientific papers compared to traditional single-score methods.
• Context-specific recommendations enabled by the system allowed users to discover papers with similar methodologies but different results, supporting analogical reasoning in research.
• Incorporating citation links into the embedding space enhanced the model’s ability to capture semantic relationships beyond lexical similarity.
• The integration of BERT-based embeddings with citation-based features led to higher precision in context-aware recommendations than using text or citation features alone.
• The system demonstrated improved performance in identifying relevant literature for expert users with complex, multi-faceted information needs.
• Evaluation showed that context-aware recommendations increased the diversity and relevance of results, particularly for cross-domain analogical queries.