[Paper Review] Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns

Xin Liu, Zheng Li+6 more|arXiv (Cornell University)|Dec 23, 2023

Recommender Systems and Techniques4 citations

TL;DR

This paper proposes FAPAT, a Frequent Attribute Pattern Augmented Transformer that enhances session-based recommendation by modeling user intent through frequent, compact attribute patterns as memory. By fusing these patterns with session sequences via gated attention and a transformer block, FAPAT improves next-item prediction and intent understanding, achieving a 4.5% average gain across Hits, NDCG, and MRR on public and industrial benchmarks.

ABSTRACT

The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history. However, constructing global or local transition graphs to supplement session data can lead to noisy correlations and user intent vanishing. In this work, we propose the Frequent Attribute Pattern Augmented Transformer (FAPAT) that characterizes user intents by building attribute transition graphs and matching attribute patterns. Specifically, the frequent and compact attribute patterns are served as memory to augment session representations, followed by a gate and a transformer block to fuse the whole session information. Through extensive experiments on two public benchmarks and 100 million industrial data in three domains, we demonstrate that FAPAT consistently outperforms state-of-the-art methods by an average of 4.5% across various evaluation metrics (Hits, NDCG, MRR). Besides evaluating the next-item prediction, we estimate the models' capabilities to capture user intents via predicting items' attributes and period-item recommendations.

Motivation & Objective

• To address the limitations of existing session-based recommendation models in capturing user intent, especially in short or noisy sessions.
• To reduce noise and over-smoothing in global transition graphs by replacing them with frequent, compact attribute patterns.
• To enhance temporal modeling by integrating attribute patterns as memory to preserve user intent evolution.
• To improve model robustness and intent understanding through attribute estimation and period-item recommendation tasks.
• To demonstrate that pattern-based augmentation outperforms global graph learning and pretraining in real-world industrial settings.

Proposed method

• Frequent attribute patterns—such as color or brand sequences—are mined from session data using graph pattern mining to form compact, high-frequency hyper-edges.
• Jaccard similarity is used to rank and retrieve the most relevant attribute patterns, reducing graph density and computational cost.
• A gate mechanism and multi-head graph attention align session sequences with learned attribute pattern representations in a shared embedding space.
• Memory-augmented self-attention with relative position bias fuses session-level actions and pattern-level signals for enhanced representation.
• The final fused representation is processed by a transformer block to model long-range dependencies and enrich temporal dynamics.
• The framework is trained end-to-end via backpropagation, with ablation studies validating each component’s contribution.

Figure 1 : Local session graph construction.

Experimental results

Research questions

• RQ1 Can frequent attribute patterns effectively capture user intent in short, anonymous sessions where global graphs fail?
• RQ2 How does pattern-based memory augmentation compare to global transition graphs in reducing noise and over-smoothing in session-based recommendation?
• RQ3 To what extent can attribute pattern modeling improve not only next-item prediction but also intent-aware tasks like attribute estimation and period-item recommendation?
• RQ4 Does the integration of attribute patterns with sequence modeling via attention lead to better generalization than pretraining or GNN-based methods?
• RQ5 Can the proposed method maintain strong performance on large-scale industrial data with 100M+ interactions?

Key findings

• FAPAT achieves an average 4.5% improvement across Hits@10, NDCG@10, and MRR@10 on both public benchmarks and three large-scale industrial datasets.
• The model outperforms state-of-the-art methods, including GraphFormer and vanilla Transformer, especially when the encoder is replaced with FAPAT’s proposed architecture.
• Attribute pattern memory significantly reduces over-smoothing and improves model robustness, particularly in sparse or noisy data regimes.
• FAPAT demonstrates superior performance in attribute estimation, achieving 90.77% Hits@10 on the Beauty dataset and 59.49% MRR on Tmall, outperforming all baselines.
• In period-item recommendation, FAPAT maintains high recall and NDCG across 3-, 5-, and 10-click horizons, indicating strong intent forecasting capability.
• Ablation studies confirm that removing attribute pattern augmentation leads to a measurable drop in performance, validating its critical role in intent modeling.

Figure 2 : A session graph enriched by multiplex attribute graphs.