[논문 리뷰] RAIE: Region-Aware Incremental Preference Editing with LoRA for LLM-based Recommendation

Jin Zeng, Yupeng Qi외 5명|arXiv (Cornell University)|2026. 02. 28.

Recommender Systems and Techniques인용 수 0

한 줄 요약

RAIE는 지역 수준의 지식 영역과 지역별 LoRA 어댑터를 도입하여 LLM 기반 추천 시스템을 변화하는 사용자 선호도에 안정적이고 효율적으로 적응시키고, 지역화된 영역만 업데이트하여 망각을 완화한다.

ABSTRACT

Large language models (LLMs) are increasingly adopted as the backbone of recommender systems. However, user-item interactions in real-world scenarios are non-stationary, making preference drift over time inevitable. Existing model update strategies mainly rely on global fine-tuning or pointwise editing, but they face two fundamental challenges: (i) imbalanced update granularity, where global updates perturb behaviors unrelated to the target while pointwise edits fail to capture broader preference shifts; (ii) unstable incremental updates, where repeated edits interfere with prior adaptations, leading to catastrophic forgetting and inconsistent recommendations. To address these issues, we propose Region-Aware Incremental Editing (RAIE), a plug-in framework that freezes the backbone model and performs region-level updates. RAIE first constructs semantically coherent preference regions via spherical k-means in the representation space. It then assigns incoming sequences to regions via confidence-aware gating and performs three localized edit operations - Update, Expand, and Add - to dynamically revise the affected region. Each region is equipped with a dedicated Low-Rank Adaptation (LoRA) module, which is trained only on the region's updated data. During inference, RAIE routes each user sequence to its corresponding region and activates the region-specific adapter for prediction. Experiments on two benchmark datasets under a time-sliced protocol that segments data into Set-up (S), Finetune (F), and Test (T) show that RAIE significantly outperforms state-of-the-art baselines while effectively mitigating forgetting. These results demonstrate that region-aware editing offers an accurate and scalable mechanism for continual adaptation in dynamic recommendation scenarios. Our code is available at https://github.com/fengaogao/RAIE.

연구 동기 및 목표

• Address non-stationary user preferences and preference drift in LLM-based recommender systems.
• Propose a plug-in region-aware editing framework that localizes updates to regions of semantic interest.
• Use sphere-k-means to form semantically coherent knowledge regions and assign a dedicated LoRA adapter per region.
• Develop region-aware routing and editing operations (Update, Expand, Add) to update only affected regions.
• Demonstrate improved retention and adaptation on MovieLens-10M and Yelp under time-sliced evaluation.

제안 방법

• Construct knowledge regions by segmenting user histories into overlapping subsequences, encoding via a frozen LLM backbone, normalizing embeddings, and clustering with spherical k-means to obtain region centroids and radii.
• Attach a region-specific LoRA adapter to the backbone for each region and train these adapters on region-specific data (S and F phases).
• During F phase, route new subsequences to candidate regions using cosine similarity with region centers, then apply region editing operations (Update, Expand, Add) to adjust region centers/radii and boundaries based on confidence scores.
• Region-aware editing rules determine when to Update an existing region, Expand its boundary, or Add a new region, guided by confidence gaps (p* and delta) and thresholds.
• Region-specific LoRA adapters are trained with a composite loss: LoRA objective on region data plus a separation penalty to reduce region overlap (L_p).
• During T phase, route each new subsequence to the most compatible region and apply the corresponding region-specific LoRA adapter for prediction.

실험 결과

연구 질문

• RQ1 How does RAIE perform compared with state-of-the-art plug-in baselines across backbone models and datasets?
• RQ2 What is the contribution of knowledge-region construction, region-aware editing, and region-specific LoRA training to overall performance?
• RQ3 How sensitive is RAIE to hyperparameters and do results generalize across datasets?
• RQ4 Does RAIE provide interpretable reflections of pre-edit and post-edit preference structure?

주요 결과

Backbone Variant	S.R@10 (MovieLens-10M)	S.N@10 (MovieLens-10M)	T.R@10 (MovieLens-10M)	T.N@10 (MovieLens-10M)
+RAIE (BERT4Rec)	0.1795	0.0957	0.0870	0.0453
+RAIE (SASRec)	0.0686	0.0327	0.0449	0.0123
+RAIE (TiSASRec)	0.1646	0.0868	0.0483	0.0135
+RAIE (OpenP5)	0.2768	0.1686	0.0935	0.0486

• RAIE consistently achieves the best predictive performance on both MovieLens-10M and Yelp under the time-sliced Set-up/Test protocol across multiple backbones.
• Region-aware editing enables strong adaptation to emerging preferences while maintaining retention of past preferences, outperforming global LoRA and replay/consistency methods.
• Region-specific LoRA adapters localize updates to drifted user interests, reducing interference with stable regions and mitigating forgetting.
• RAIE demonstrates superior T (Test) metrics while maintaining competitive S (Set-up) metrics across backbones like BERT4Rec, SASRec, TiSASRec, and OpenP5.
• Experiments confirm that region-aware routing and localized editing provide effective continual adaptation in dynamic recommendation scenarios.