[論文レビュー] Data-efficient Fine-tuning for LLM-based Recommendation

Xinyu Lin, Wenjie Wang+5 more|arXiv (Cornell University)|Jan 30, 2024

Web Data Mining and Analysis被引用数 5

ひとこと要約

The paper proposes DEALRec, a data pruning method that selects influential samples for few-shot fine-tuning of LLM-based recommender models using an influence score (computed via influence functions with Hessian-vector products) and an effort score to bridge surrogate models and LLMs, achieving high accuracy with a tiny subset (as low as 2%) and large time-cost reductions.

ABSTRACT

Leveraging Large Language Models (LLMs) for recommendation has recently garnered considerable attention, where fine-tuning plays a key role in LLMs' adaptation. However, the cost of fine-tuning LLMs on rapidly expanding recommendation data limits their practical application. To address this challenge, few-shot fine-tuning offers a promising approach to quickly adapt LLMs to new recommendation data. We propose the task of data pruning for efficient LLM-based recommendation, aimed at identifying representative samples tailored for LLMs' few-shot fine-tuning. While coreset selection is closely related to the proposed task, existing coreset selection methods often rely on suboptimal heuristic metrics or entail costly optimization on large-scale recommendation data. To tackle these issues, we introduce two objectives for the data pruning task in the context of LLM-based recommendation: 1) high accuracy aims to identify the influential samples that can lead to high overall performance; and 2) high efficiency underlines the low costs of the data pruning process. To pursue the two objectives, we propose a novel data pruning method based on two scores, i.e., influence score and effort score, to efficiently identify the influential samples. Particularly, the influence score is introduced to accurately estimate the influence of sample removal on the overall performance. To achieve low costs of the data pruning process, we use a small-sized surrogate model to replace LLMs to obtain the influence score. Considering the potential gap between the surrogate model and LLMs, we further propose an effort score to prioritize some hard samples specifically for LLMs. Empirical results on three real-world datasets validate the effectiveness of our proposed method. In particular, the proposed method uses only 2% samples to surpass the full data fine-tuning, reducing time costs by 97%.

研究の動機と目的

• Motivate the need for data-efficient fine-tuning of LLM-based recommender systems due to high costs and continuous data growth.
• Define data pruning for LLM-based recommendations as selecting a representative subset tailored for few-shot fine-tuning.
• Develop DEALRec to identify influential samples via an influence score and mitigate surrogate-LLM gap with an effort score.
• Demonstrate that data pruning with DEALRec enables competitive or superior performance with substantial efficiency gains across real-world datasets.

提案手法

• Define two optimization objectives for data pruning: high accuracy (low empirical risk) and high efficiency (low pruning costs).
• Introduce DEALRec with two scores: influence score (estimating a sample's impact on empirical risk via influence functions and second-order optimization) and effort score (measuring LLM learning effort for a sample).
• Compute influence score using a surrogate model to avoid costly LLM retraining and a symmetric formulation to reuse a single Hessian inverse-vector product estimation.
• Regularize the influence score with the effort score to bridge the gap between surrogate models and LLMs (gap regularization).
• Use stratified (coverage-enhanced) sampling to select a diverse and representative subset for few-shot LLM fine-tuning.
• Instantiate DEALRec on two LLM-based recommender backends and validate on three real-world datasets.

実験結果

リサーチクエスチョン

• RQ1 RQ1: How does DEALRec compare with coreset baselines and full-data training for LLM-based recommendations?
• RQ2 RQ2: How do DEALRec components (influence score, gap regularization, stratified sampling) affect performance and generalizability across surrogate models?
• RQ3 RQ3: How does DEALRec perform across different selection ratios and how does it affect overall effectiveness?

主な発見

• DEALRec achieves strong data-efficiency, surpassing full-data fine-tuning using as few as 2% of the data in some settings.
• Empirical results on three real-world datasets show DEALRec improving efficiency without sacrificing accuracy relative to baselines.
• The method integrates influence-based sample ranking with gap regularization to compensate for surrogate-LLM differences.
• Stratified sampling ensures data coverage and stable empirical risk bounds during few-shot fine-tuning.
• Using a surrogate model to estimate influence significantly reduces computation compared to directly using LLMs.