[Paper Review] Hierarchical Gating Networks for Sequential Recommendation

Chen Ma, Peng Kang+1 more|arXiv (Cornell University)|Jun 21, 2019

Recommender Systems and Techniques39 references4 citations

TL;DR

This paper proposes Hierarchical Gating Networks (HGN), a sequential recommendation model that jointly captures long-term and short-term user interests via feature and instance gating mechanisms, while explicitly modeling item-item relationships through a product module. Evaluated on five real-world datasets, HGN outperforms state-of-the-art methods in Top-N recommendation accuracy and achieves faster training speed due to its parameter-efficient design.

ABSTRACT

The chronological order of user-item interactions is a key feature in many recommender systems, where the items that users will interact may largely depend on those items that users just accessed recently. However, with the tremendous increase of users and items, sequential recommender systems still face several challenging problems: (1) the hardness of modeling the long-term user interests from sparse implicit feedback; (2) the difficulty of capturing the short-term user interests given several items the user just accessed. To cope with these challenges, we propose a hierarchical gating network (HGN), integrated with the Bayesian Personalized Ranking (BPR) to capture both the long-term and short-term user interests. Our HGN consists of a feature gating module, an instance gating module, and an item-item product module. In particular, our feature gating and instance gating modules select what item features can be passed to the downstream layers from the feature and instance levels, respectively. Our item-item product module explicitly captures the item relations between the items that users accessed in the past and those items users will access in the future. We extensively evaluate our model with several state-of-the-art methods and different validation metrics on five real-world datasets. The experimental results demonstrate the effectiveness of our model on Top-N sequential recommendation.

Motivation & Objective

• To address the challenge of modeling both long-term and short-term user interests from sparse implicit feedback in sequential recommendation.
• To overcome limitations in existing models that fail to selectively attend to informative item features and important items in short-term sequences.
• To explicitly capture item-item relationships, as users often interact with related items in succession.
• To design a parameter-efficient architecture that maintains high performance while reducing model complexity compared to RNNs and CNNs.
• To improve training efficiency without sacrificing recommendation accuracy, especially in large-scale settings.

Proposed method

• The model employs a feature gating module that adaptively selects relevant latent features of items based on user preference, enabling user-specific feature routing.
• An instance gating module identifies and prioritizes key items in the user's recent interaction history to reflect short-term interests.
• The hierarchical gating mechanism forms a two-level routing system: features are selected first, then critical instances are passed to downstream layers.
• An item-item product module explicitly models relationships between past items and future candidate items, enhancing sequential pattern learning.
• The model integrates with Bayesian Personalized Ranking (BPR) for end-to-end optimization using pairwise ranking loss.
• The architecture avoids complex RNNs or CNNs, relying instead on parameter-efficient matrix operations with O(N²d + Nd²) complexity, where N is the number of items and d is embedding size.

Experimental results

Research questions

• RQ1 Can a parameter-efficient architecture effectively model both long-term and short-term user interests in sequential recommendation?
• RQ2 How does feature-level and instance-level gating improve the representation of user preferences compared to standard RNN or CNN models?
• RQ3 To what extent can explicit modeling of item-item relationships enhance sequential recommendation performance?
• RQ4 How does the proposed model compare in training efficiency to existing deep learning-based sequential models like SASRec and Caser?
• RQ5 What is the optimal configuration of hyperparameters such as embedding dimension and sequence length for maximizing recommendation accuracy?

Key findings

• HGN achieves state-of-the-art performance on five real-world datasets (Books, Comics, CDs, Children, ML20M) across multiple evaluation metrics, including R@5 and R@10.
• The model reduces the number of learnable parameters to 5,350 (with d=50), significantly fewer than GRU (15,300) and CNN (26,154) baselines, indicating high parameter efficiency.
• HGN demonstrates the fastest training speed among all compared models, with inference times of 0.957s per epoch on the CDs dataset and under 3s on most others.
• The inclusion of the item-item product module leads to consistent performance gains, confirming that modeling item relationships improves sequential prediction.
• Performance improves with increasing embedding dimension d up to a point, after which gains plateau, indicating sufficient capacity is reached around d=50.
• Optimal performance is achieved with |L|=3 and |T|=3, suggesting that a short history of three items and prediction of three future items yields the best balance of relevance and noise reduction.