[Paper Review] Towards Effective Research-Paper Recommender Systems and User Modeling based on Mind Maps

Joeran Beel|arXiv (Cornell University)|Mar 27, 2017

Mind wandering and attention17 citations

TL;DR

This paper proposes a novel user modeling approach for research-paper recommender systems using mind maps as input. By analyzing node structure, relations, and temporal metadata in Docear's mind maps, the system achieves a 7.20% click-through rate—nearly double the best baseline—demonstrating that mind map-based modeling is as effective as modeling based on downloaded or cited papers.

ABSTRACT

While user-modeling and recommender systems successfully utilize items like emails, news, and movies, they widely neglect mind-maps as a source for user modeling. We consider this a serious shortcoming since we assume user modeling based on mind maps to be equally effective as user modeling based on other items. Hence, millions of mind-mapping users could benefit from user-modeling applications such as recommender systems. The objective of this doctoral thesis is to develop an effective user-modeling approach based on mind maps. To achieve this objective, we integrate a recommender system in our mind-mapping and reference-management software Docear. The recommender system builds user models based on the mind maps, and recommends research papers based on the user models. As part of our research, we identify several variables relating to mind-map-based user modeling, and evaluate the variables' impact on user-modeling effectiveness with an offline evaluation, a user study, and an online evaluation based on 430,893 recommendations displayed to 4,700 users. We find, among others, that the number of analyzed nodes, modification time, visibility of nodes, relations between nodes, and number of children and siblings of a node affect the effectiveness of user modeling. When all variables are combined in a favorable way, this novel approach achieves click-through rates of 7.20%, which is nearly twice as effective as the best baseline. In addition, we show that user modeling based on mind maps performs about as well as user modeling based on other items, namely the research articles users downloaded or cited. Our findings let us to conclude that user modeling based on mind maps is a promising research field, and that developers of mind-mapping applications should integrate recommender systems into their applications. Such systems could create additional value for millions of mind-mapping users.

Motivation & Objective

• To develop an effective user-modeling approach based on mind maps for research-paper recommendation.
• To evaluate the impact of various mind map structural and temporal variables on user-modeling effectiveness.
• To compare the performance of mind map-based user modeling against traditional methods using downloaded or cited papers.
• To integrate a recommender system into Docear that leverages mind map data for personalized recommendations.
• To demonstrate that mind map-based user modeling is a viable and effective alternative to existing modeling techniques.

Proposed method

• The system extracts user models from mind maps in Docear, using structural features such as node depth, number of children and siblings, and visibility of nodes.
• It incorporates temporal metadata, including modification time and creation time, to weight nodes based on recency and engagement.
• Relations between nodes are analyzed to capture semantic and conceptual relationships in the user's knowledge structure.
• A hybrid recommendation algorithm combines these features to generate personalized paper suggestions based on the user's mind map structure.
• Offline, user study, and online evaluations were conducted using 430,893 recommendations across 4,700 users to assess effectiveness.
• The approach uses click-through rate as the primary evaluation metric to compare performance against baseline methods.

Experimental results

Research questions

• RQ1 How do structural properties of mind maps—such as node depth, number of children, and sibling count—affect user-modeling effectiveness?
• RQ2 To what extent does the modification time of nodes influence the accuracy of user modeling in recommendation systems?
• RQ3 How do visibility and relational connections between nodes impact recommendation relevance?
• RQ4 Can user modeling based on mind maps achieve performance comparable to modeling based on downloaded or cited research papers?
• RQ5 What combination of mind map features yields the highest click-through rate in a real-world recommendation setting?

Key findings

• The proposed mind map-based user modeling approach achieved a click-through rate of 7.20%, nearly twice that of the best baseline method.
• The number of analyzed nodes, modification time, visibility of nodes, and relations between nodes significantly influenced recommendation effectiveness.
• The number of children and siblings of a node were found to be strong predictors of user modeling quality.
• User modeling based on mind maps performed comparably to modeling based on downloaded or cited research articles in terms of recommendation effectiveness.
• Combining structural and temporal features in a favorable configuration led to the highest performance, validating the approach’s robustness.
• The results suggest that mind map-based user modeling is a viable and effective alternative for recommender systems in academic contexts.