[Paper Review] PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

Jianxin Chang, Chenbin Zhang+5 more|arXiv (Cornell University)|Feb 2, 2023

Recommender Systems and Techniques4 citations

TL;DR

PEPNet is a plug-and-play neural network that enhances multi-domain and multi-task recommendation by dynamically personalizing both embedding features and deep neural network parameters using gate mechanisms. It achieves over 1% online improvement across multiple metrics in Kuaishou's production system, serving over 300 million users daily.

ABSTRACT

With the increase of content pages and interactive buttons in online services such as online-shopping and video-watching websites, industrial-scale recommender systems face challenges in multi-domain and multi-task recommendations. The core of multi-task and multi-domain recommendation is to accurately capture user interests in multiple scenarios given multiple user behaviors. In this paper, we propose a plug-and-play extit{ extbf{P}arameter and extbf{E}mbedding extbf{P}ersonalized extbf{Net}work ( extbf{PEPNet})} for multi-domain and multi-task recommendation. PEPNet takes personalized prior information as input and dynamically scales the bottom-level Embedding and top-level DNN hidden units through gate mechanisms. extit{Embedding Personalized Network (EPNet)} performs personalized selection on Embedding to fuse features with different importance for different users in multiple domains. extit{Parameter Personalized Network (PPNet)} executes personalized modification on DNN parameters to balance targets with different sparsity for different users in multiple tasks. We have made a series of special engineering optimizations combining the Kuaishou training framework and the online deployment environment. By infusing personalized selection of Embedding and personalized modification of DNN parameters, PEPNet tailored to the interests of each individual obtains significant performance gains, with online improvements exceeding 1\% in multiple task metrics across multiple domains. We have deployed PEPNet in Kuaishou apps, serving over 300 million users every day.

Motivation & Objective

• Address the imperfectly double seesaw problem in industrial recommender systems, where data sparsity and domain/task distribution shifts co-occur.
• Enable effective joint learning across multiple domains and multiple tasks without compromising model efficiency or deployment cost.
• Introduce a plug-and-play architecture that infuses personalized prior information into both embedding and parameter spaces.
• Overcome limitations of existing multi-domain and multi-task learning methods that fail to handle semantic heterogeneity and target sparsity simultaneously.
• Optimize for real-world deployment by integrating with Kuaishou’s training and inference pipeline, ensuring low-latency inference at scale.

Proposed method

• Propose a dual-branch architecture: Embedding Personalized Network (EPNet) and Parameter Personalized Network (PPNet), each using learnable gate mechanisms.
• EPNet applies user-specific gates to dynamically scale embedding features based on their relevance to individual users, enabling personalized feature fusion.
• PPNet uses task- and user-specific gates to modulate DNN hidden unit activations, adapting model parameters per user and task to balance sparsity.
• Integrate personalized prior information (e.g., user behavior history) as input to the gate networks, enabling dynamic adaptation per user.
• Design a modular, plug-and-play structure compatible with existing recommendation models, minimizing retraining and deployment overhead.
• Apply engineering optimizations for efficient training and online inference, including quantization-aware training and model partitioning in Kuaishou’s framework.

Experimental results

Research questions

• RQ1 How can we effectively model heterogeneous feature semantics and importance across multiple domains while handling task-specific sparsity?
• RQ2 Can personalized gating mechanisms improve performance in multi-domain and multi-task recommendation without increasing model complexity?
• RQ3 How does the integration of personalized prior information into both embedding and parameter spaces affect model generalization and online metrics?
• RQ4 What is the impact of dynamic parameter and embedding personalization on the double seesaw phenomenon in large-scale industrial recommender systems?
• RQ5 Can a plug-and-play architecture achieve significant performance gains while maintaining low inference latency in real-world deployment?

Key findings

• PEPNet achieves online performance gains exceeding 1% across multiple key metrics (e.g., CTR, engagement) in Kuaishou’s multi-domain, multi-task recommendation system.
• The model significantly reduces the double seesaw effect by enabling personalized feature selection and parameter adaptation across domains and tasks.
• Embedding Personalization via EPNet improves feature fusion by weighting embeddings based on user-specific relevance, enhancing representation quality.
• Parameter Personalization via PPNet effectively balances sparse and dense prediction targets by adapting DNN weights per user and task.
• The plug-and-play design enables seamless integration into Kuaishou’s existing training and serving pipeline, with minimal engineering overhead.
• Offline and online evaluations confirm consistent improvements across diverse recommendation scenarios, including short-video discovery and e-commerce-like interactions.