[Paper Review] User-Sensitive Recommendation Ensemble with Clustered Multi-Task Learning

Menghan Wang, Xiaolin Zheng+1 more|arXiv (Cornell University)|Apr 28, 2018

Recommender Systems and Techniques14 references3 citations

TL;DR

This paper proposes UREC, a user-sensitive recommendation ensemble method that clusters users based on prediction patterns and applies clustered multi-task learning to learn personalized ensemble weights, significantly reducing user-skewed prediction bias. UREC outperforms traditional ensembles and base models, achieving a 79.11% winning rate on MovieLens-100K and demonstrating robustness to user heterogeneity and new user challenges.

ABSTRACT

This paper considers recommendation algorithm ensembles in a user-sensitive manner. Recently researchers have proposed various effective recommendation algorithms, which utilized different aspects of the data and different techniques. However, the "user skewed prediction" problem may exist for almost all recommendation algorithms -- algorithms with best average predictive accuracy may cover up that the algorithms may perform poorly for some part of users, which will lead to biased services in real scenarios. In this paper, we propose a user-sensitive ensemble method named "UREC" to address this issue. We first cluster users based on the recommendation predictions, then we use multi-task learning to learn the user-sensitive ensemble function for the users. In addition, to alleviate the negative effects of new user problem to clustering users, we propose an approximate approach based on a spectral relaxation. Experiments on real-world datasets demonstrate the superiority of our methods.

Motivation & Objective

• Address the user-skewed prediction problem in recommendation systems, where top-performing models fail for subsets of users despite strong average performance.
• Overcome the limitations of traditional ensemble methods that apply uniform weights across all users, ignoring user heterogeneity.
• Develop a scalable and reliable ensemble strategy that groups users into homogeneous clusters based on algorithmic prediction patterns.
• Mitigate the negative impact of new users on user clustering by introducing a spectral relaxation-based approximation method (UREC appr).

Proposed method

• Cluster users into homogeneous groups using k-medoids based on the prediction differences of base recommendation algorithms (e.g., KNN, MF, SVD++).
• Formulate the ensemble learning as a clustered multi-task learning problem, where each cluster shares a common set of ensemble weights.
• Use a regularized multi-task learning framework to jointly optimize ensemble weights across clusters, improving generalization and reducing overfitting.
• Apply spectral relaxation to the clustering objective to create UREC appr, a computationally efficient approximation that handles new users effectively.
• Leverage user-level performance metrics (e.g., RMSE) to define cluster similarity and guide the clustering process.
• Train the ensemble model using a convex relaxation of the clustering objective, enabling faster convergence and scalability.

Experimental results

Research questions

• RQ1 Can clustering users based on their prediction patterns improve ensemble fairness and reduce user-skewed performance?
• RQ2 How does clustered multi-task learning compare to traditional ensemble methods in handling user heterogeneity?
• RQ3 To what extent does the proposed spectral relaxation (UREC appr) mitigate the new user problem in user clustering?
• RQ4 Does the user-sensitive ensemble strategy significantly reduce variance in user-level performance compared to baseline models?
• RQ5 Can the proposed method achieve higher winning rates across users than individual models and standard stacking approaches?

Key findings

• UREC achieves a 79.11% winning rate on the MovieLens-100K dataset, outperforming Stacking user (79.11%) and Stacking one (67.32%), indicating strong user-level fairness.
• On MovieLens-100K, UREC reduces the variance of RMSE across users to 0.0263, significantly lower than SVD++ (0.0539), demonstrating improved consistency.
• UREC appr achieves a 67.29% winning rate on MovieLens-1M, outperforming SVD++ (58.81%) and Stacking one (69.22%), showing strong generalization.
• The runtime of UREC appr is 265 seconds on MovieLens-100K, significantly faster than UREC (452 seconds), confirming improved scalability.
• UREC achieves a 78.47% winning rate against base models on MovieLens-100K, indicating consistent superiority across diverse user groups.
• The spectral relaxation in UREC appr enables effective handling of new users, making it suitable for online recommendation systems with cold-start challenges.