[Paper Review] Recommender System for Online Dating Service

Lukas Brozovsky, V. Petřı́ček|ArXiv.org|Mar 9, 2007

Recommender Systems and TechniquesComputer Science17 references117 citations

TL;DR

This paper proposes a collaborative filtering (CF)-based recommender system for online dating services to combat information overload and improve match quality. Using real-world data from Libimseti.cz, it demonstrates that user-user and item-item CF algorithms significantly outperform global popularity (mean) and random baselines in prediction accuracy (NMAE: 3.08% and 2.04%) and are preferred by users in blind experiments, with 64.38% of duels won over the mean algorithm.

ABSTRACT

Users of online dating sites are facing information overload that requires them to manually construct queries and browse huge amount of matching user profiles. This becomes even more problematic for multimedia profiles. Although matchmaking is frequently cited as a typical application for recommender systems, there is a surprising lack of work published in this area. In this paper we describe a recommender system we implemented and perform a quantitative comparison of two collaborative filtering (CF) and two global algorithms. Results show that collaborative filtering recommenders significantly outperform global algorithms that are currently used by dating sites. A blind experiment with real users also confirmed that users prefer CF based recommendations to global popularity recommendations. Recommender systems show a great potential for online dating where they could improve the value of the service to users and improve monetization of the service.

Motivation & Objective

• To address information overload in online dating, where users face excessive profile browsing due to poor ranking and query limitations.
• To evaluate the effectiveness of collaborative filtering (CF) versus global popularity algorithms in recommending dating profiles.
• To assess user preference between CF-based and globally popular recommendations in a real-world setting.
• To demonstrate that CF-based recommenders can significantly improve matchmaking quality and service value.

Proposed method

• The study uses a real-world dataset from the dating site Libimseti.cz, containing 14,057 ratings across 1,000 profiles with 11.39‰ sparsity.
• Four algorithms are benchmarked: Random (uniform random prediction), Mean (global popularity), User-User CF (k-nearest neighbors), and Item-Item CF (cosine similarity-based).
• Prediction accuracy is measured using Normalized Mean Absolute Error (NMAE), with cross-validation on a snapshot of the dataset.
• A blind user experiment with 111 participants compared recommendation lists from User-User (10,50), Mean, and Random algorithms, using a 'duel' format to assess preference.
• The user interface ensured recommendations were for the opposite sex and excluded already-rated profiles to avoid bias.
• Index structures were evaluated for scalability, confirming ColFi’s ability to handle real-world traffic levels.

Experimental results

Research questions

• RQ1 Can collaborative filtering algorithms significantly improve recommendation accuracy over global popularity baselines in online dating?
• RQ2 Do users perceive CF-based recommendations as higher quality than globally popular or random recommendations?
• RQ3 How does the performance of user-user and item-item CF algorithms compare to mean and random baselines in terms of NMAE?
• RQ4 What is the impact of user interface design on rating truthfulness and recommendation quality in dating platforms?
• RQ5 Can hybrid or reciprocal matching models further improve recommendation relevance in dating systems?

Key findings

• User-User CF achieved the lowest NMAE of 3.08%, outperforming the Item-Item CF (2.04%) and the global Mean algorithm.
• The Item-Item CF algorithm achieved the best prediction accuracy with an NMAE of 2.04%, significantly lower than the Mean algorithm’s 3.95%.
• In user preference experiments, the User-User algorithm won 64.38% of duels against the Mean algorithm, indicating strong user preference for CF-based recommendations.
• The Mean algorithm won 35.62% of duels against User-User, suggesting a universal preference for popular profiles, but still losing overall.
• The Random algorithm performed poorly, losing 87.50% of duels to User-User, confirming its irrelevance as a recommendation baseline.
• The study confirms that CF-based recommenders are not only more accurate but also more preferred by real users, supporting their deployment in real dating platforms.