[Paper Review] Learning to Recommend via Meta Parameter Partition

Liang Zhao, Yang Wang+2 more|arXiv (Cornell University)|Dec 4, 2019

Recommender Systems and Techniques38 references4 citations

TL;DR

This paper proposes a two-stage meta-learning approach that partitions model parameters into fixed (user-invariant) and adaptive (user-specific) components to efficiently address user cold start in recommendation systems. By decoupling parameter learning, the method reduces storage and computation costs while enabling continual adaptation; experiments show a 4.94% absolute AUC improvement over baselines, with 2.48% from offline meta-training and an additional 2.46% gain from online meta-finetuning.

ABSTRACT

In this paper we propose to solve an important problem in recommendation -- user cold start, based on meta leaning method. Previous meta learning approaches finetune all parameters for each new user, which is both computing and storage expensive. In contrast, we divide model parameters into fixed and adaptive parts and develop a two-stage meta learning algorithm to learn them separately. The fixed part, capturing user invariant features, is shared by all users and is learned during offline meta learning stage. The adaptive part, capturing user specific features, is learned during online meta learning stage. By decoupling user invariant parameters from user dependent parameters, the proposed approach is more efficient and storage cheaper than previous methods. It also has potential to deal with catastrophic forgetting while continually adapting for streaming coming users. Experiments on production data demonstrates that the proposed method converges faster and to a better performance than baseline methods. Meta-training without online meta model finetuning increases the AUC from 72.24% to 74.72% (2.48% absolute improvement). Online meta training achieves a further gain of 2.46\% absolute improvement comparing with offline meta training.

Motivation & Objective

• Address the user cold start problem in personalized recommendation systems where new users have minimal interaction history.
• Reduce computational and storage overhead in meta-learning-based recommendation by decoupling user-invariant and user-specific model parameters.
• Enable continual adaptation to streaming users while mitigating catastrophic forgetting through parameter partitioning.
• Improve recommendation performance in few-shot settings using offline and online meta-learning stages.

Proposed method

• Partition model parameters into fixed (shared across users) and adaptive (user-specific) components to separate user-invariant and user-dependent features.
• Train the fixed parameters via offline meta-learning on historical user data to learn a robust initialization.
• Fine-tune the adaptive parameters online using each new user’s limited interaction data to personalize the model.
• Apply a two-stage training process: first, meta-train the fixed parameters; second, perform online meta-finetuning of adaptive parameters per new user.
• Fix middle hidden layers (2nd–4th) during online adaptation, while allowing the embedding and classifier layers to be updated for user-specific adaptation.
• Use AUC as the primary evaluation metric to assess CTR prediction performance across offline and online stages.

Experimental results

Research questions

• RQ1 Can parameter partitioning into fixed and adaptive components improve efficiency and performance in cold-start recommendation?
• RQ2 How does decoupled meta-learning compare to full-parameter fine-tuning in terms of convergence speed and AUC performance?
• RQ3 To what extent does online meta-finetuning with incremental user data improve model performance beyond offline meta-training?
• RQ4 Which network layers should be kept fixed during online adaptation to balance generalization and personalization?

Key findings

• The proposed method achieved a 4.94% absolute improvement in AUC, rising from 72.24% (baseline) to 77.18%.
• Offline meta-training alone improved AUC by 2.48% (to 74.72%), demonstrating the benefit of meta-initialization over standard joint training.
• Online meta-finetuning provided an additional 2.46% absolute AUC gain over offline meta-training, showing consistent improvement with more user data.
• Fixing the 2nd and 3rd hidden layers during online adaptation yielded the highest performance, indicating these layers capture user-invariant features.
• The model converged faster and achieved better performance than both conventional meta-learning and baseline methods, especially on users with only 8–10 interaction records.
• The ablation study confirmed that meta-initialization, parameter decoupling, and online learning all contributed significantly to the final performance gain.