[Paper Review] LKPNR: LLM and KG for Personalized News Recommendation Framework

Hao Chen, Xie Runfeng+5 more|arXiv (Cornell University)|Aug 23, 2023

Topic Modeling7 citations

TL;DR

This paper proposes LKPNR, a novel personalized news recommendation framework that integrates Large Language Models (LLM) for deep semantic understanding of news text and Knowledge Graphs (KG) to capture multi-hop structural relationships among news entities. By fusing LLM-generated contextual representations with KG-augmented entity features through multi-head attention, LKPNR significantly improves recommendation performance, especially for long-tail items, outperforming traditional models on the MIND dataset.

ABSTRACT

Accurately recommending candidate news articles to users is a basic challenge faced by personalized news recommendation systems. Traditional methods are usually difficult to grasp the complex semantic information in news texts, resulting in unsatisfactory recommendation results. Besides, these traditional methods are more friendly to active users with rich historical behaviors. However, they can not effectively solve the "long tail problem" of inactive users. To address these issues, this research presents a novel general framework that combines Large Language Models (LLM) and Knowledge Graphs (KG) into semantic representations of traditional methods. In order to improve semantic understanding in complex news texts, we use LLMs' powerful text understanding ability to generate news representations containing rich semantic information. In addition, our method combines the information about news entities and mines high-order structural information through multiple hops in KG, thus alleviating the challenge of long tail distribution. Experimental results demonstrate that compared with various traditional models, the framework significantly improves the recommendation effect. The successful integration of LLM and KG in our framework has established a feasible path for achieving more accurate personalized recommendations in the news field. Our code is available at https://github.com/Xuan-ZW/LKPNR.

Motivation & Objective

• To address the limitations of traditional news recommendation models in capturing complex semantic features from news text.
• To alleviate the long-tail problem by improving representation learning for less popular, inactive-user-targeted news items.
• To integrate Large Language Models (LLM) and Knowledge Graphs (KG) into a unified framework for enhanced news and user representation.
• To enable flexible integration with existing general news recommendation models through modular design.
• To improve recommendation accuracy by leveraging both contextual semantics from LLMs and structural relationships from KGs.

Proposed method

• Utilizes a Large Language Model (LLM) to generate rich, context-aware semantic representations of news titles and abstracts, capturing deep linguistic and world knowledge.
• Employs a KG-Augmented Encoder that performs multi-hop knowledge graph inference to extract high-order structural information from news entities.
• Applies multi-head query projection to dynamically weigh neighboring entities in the KG, enabling diverse and collaborative information integration.
• Fuses LLM-generated news representations with KG-augmented entity features using attention mechanisms to enhance overall news representation.
• Supports plug-and-play integration with various general news recommendation models by serving as a semantic and structural enhancement module.
• Uses a retrieval-based query mechanism where LLM-mapped queries outperform standard encoder queries in retrieving relevant KG neighbors.

Figure 1: Differences between traditional news recommendation model (a) and LKPNR framework (b)

Experimental results

Research questions

• RQ1 Can LLMs improve the semantic richness of news representations beyond traditional text encoders?
• RQ2 Can multi-hop knowledge graph reasoning enhance the representation of long-tail news items in personalized recommendation?
• RQ3 How does the integration of LLM and KG affect recommendation performance compared to standalone models?
• RQ4 Does the use of multi-head attention in KG querying improve the model's ability to capture collaborative signals from neighboring entities?
• RQ5 Can the proposed framework be flexibly combined with existing general news recommendation models to boost performance?

Key findings

• The LLM-mapped query outperforms the general news encoder query in retrieving relevant KG entities, demonstrating superior semantic alignment.
• Using three attention heads (num_head=3) in the KG-Augmented Encoder yields the highest retrieval performance, indicating optimal representation space expansion.
• The integration of LLM and KG significantly improves recommendation performance on the MIND dataset, surpassing traditional baseline models.
• The framework effectively captures latent connections between candidate news and user history through KG neighbors, even when textual similarity is low.
• Visualization results confirm that candidate news with low textual correlation to user history can still achieve high matching scores when high-relevance KG neighbors are present.
• The model reduces the long-tail problem by enhancing representation learning for less popular news items through structural graph information.

Figure 2: The framework of LKPNR. The lower left corner of figure is the input news data into LKPNR, where the words marked in yellow indicates the entities. Candidate news is encoded by LK-Aug news encoder that combines LLM and KG with traditional general news encoder to obtain news representation.