[Paper Review] A Content-Driven Micro-Video Recommendation Dataset at Scale

Yongxin Ni, Yu Cheng+6 more|arXiv (Cornell University)|Sep 27, 2023

Recommender Systems and TechniquesComputer Science3 citations

TL;DR

This paper introduces MicroLens, the largest publicly available micro-video recommendation dataset with one billion user-item interactions, 34 million users, and one million videos enriched with raw modalities (text, images, audio, video). It enables end-to-end learning of video representations for recommendation, outperforming ID-based models and demonstrating that raw video features can effectively replace item IDs, especially in cold-start and popular-item settings.

ABSTRACT

Micro-videos have recently gained immense popularity, sparking critical research in micro-video recommendation with significant implications for the entertainment, advertising, and e-commerce industries. However, the lack of large-scale public micro-video datasets poses a major challenge for developing effective recommender systems. To address this challenge, we introduce a very large micro-video recommendation dataset, named "MicroLens", consisting of one billion user-item interaction behaviors, 34 million users, and one million micro-videos. This dataset also contains various raw modality information about videos, including titles, cover images, audio, and full-length videos. MicroLens serves as a benchmark for content-driven micro-video recommendation, enabling researchers to utilize various modalities of video information for recommendation, rather than relying solely on item IDs or off-the-shelf video features extracted from a pre-trained network. Our benchmarking of multiple recommender models and video encoders on MicroLens has yielded valuable insights into the performance of micro-video recommendation. We believe that this dataset will not only benefit the recommender system community but also promote the development of the video understanding field. Our datasets and code are available at https://github.com/westlake-repl/MicroLens.

Motivation & Objective

• Address the lack of large-scale, diverse, and high-quality public micro-video datasets for recommendation research.
• Provide a benchmark dataset with rich raw video modalities (text, images, audio, full video) to support content-driven recommendation models.
• Enable end-to-end training of recommender systems that jointly optimize video encoders and recommendation models.
• Investigate the performance gap between current video understanding and video recommendation, and explore the potential of video representations in improving recommendation accuracy.
• Promote the development of universal, foundation-style recommender models by providing a pre-training resource for vision-based recommendation.

Proposed method

• Collected 400,000 seed micro-videos from a social entertainment platform using homepage refreshes to ensure diversity.
• Filtered videos based on engagement (≥10,000 likes or ≥100 comments) to ensure quality and popularity.
• Acquired full modalities: video titles (text), cover images, audio clips, and raw video files for each video.
• Expanded the dataset by crawling external links from video pages to enrich content diversity.
• Constructed a final dataset of 1 million videos with 1 billion user-item interactions, using comment behavior as a proxy for user preference.
• Benchmarked over 10 recommender models and video encoders, including novel end-to-end (E2E) training setups that jointly optimize video encoders and recommendation heads.

Experimental results

Research questions

• RQ1 Can end-to-end training of video encoders and recommender models lead to superior recommendation performance compared to using pre-extracted features?
• RQ2 How does recommendation accuracy using raw video features compare to that using only cover images or item IDs?
• RQ3 Can video-based recommendation models outperform ID-based models even in highly popular item settings?
• RQ4 To what extent do raw video representations improve performance on cold-start items compared to traditional ID-based or image-based models?
• RQ5 What is the potential of MicroLens as a pre-training dataset for foundation models in video understanding and recommendation?

Key findings

• The proposed VideoRec model, which learns representations directly from raw video, consistently outperforms IDRec and ImageRec baselines across all evaluation metrics.
• End-to-end training of video encoders and recommender models achieves state-of-the-art performance, demonstrating the value of joint optimization.
• VideoRec surpasses IDRec even in warm item settings, indicating that raw video features can effectively replace item IDs in practical recommendation systems.
• VideoRec shows significant improvements on cold items, with even greater gains observed in cold-start scenarios (as shown in Appendix Figure 6).
• Using cover images alone (ImageRec) yields lower performance than using full video features (VideoRec), highlighting the importance of temporal and motion information in video understanding.
• MicroLens contains sufficient scale and diversity to support pre-training of foundation models, with potential for generating trillions of user-video interactions through frame-level sampling.