[Paper Review] Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering

Ruining He, Julian McAuley|arXiv (Cornell University)|Feb 4, 2016

Generative Adversarial Networks and Image SynthesisComputer Science36 references203 citations

TL;DR

This paper develops temporally evolving, visually-aware one-class collaborative filtering models for fashion recommendations by incorporating deep CNN visual features, epoch-based fashion evolution, and lightweight non-visual dynamics within a Bayesian Personalized Ranking framework.

ABSTRACT

Building a successful recommender system depends on understanding both the dimensions of people's preferences as well as their dynamics. In certain domains, such as fashion, modeling such preferences can be incredibly difficult, due to the need to simultaneously model the visual appearance of products as well as their evolution over time. The subtle semantics and non-linear dynamics of fashion evolution raise unique challenges especially considering the sparsity and large scale of the underlying datasets. In this paper we build novel models for the One-Class Collaborative Filtering setting, where our goal is to estimate users' fashion-aware personalized ranking functions based on their past feedback. To uncover the complex and evolving visual factors that people consider when evaluating products, our method combines high-level visual features extracted from a deep convolutional neural network, users' past feedback, as well as evolving trends within the community. Experimentally we evaluate our method on two large real-world datasets from Amazon.com, where we show it to outperform state-of-the-art personalized ranking measures, and also use it to visualize the high-level fashion trends across the 11-year span of our dataset.

Motivation & Objective

• Model users' fashion-aware personalized rankings from implicit feedback (purchases).
• Capture evolving visual factors driving fashion choices over time.
• Disentangle visual evolution from non-visual temporal dynamics for better recommendations and interpretation.
• Enable scalable training on large fashion catalogs with sparse, long-tail items.

Proposed method

• Extend matrix factorization with visual interaction: x_hat = alpha + beta_u + beta_i + <gamma_u, gamma_i> + <theta_u, theta_i>.
• Embed high-dimensional CNN image features f_i into a K'-dimensional visual space via E: theta_i = E f_i.
• Model temporal evolution through time-dependent components: E(t) = E + Delta_E(t); theta_i(t) = E(t) f_i and theta_u(t) as needed.
• Introduce temporal weighting and epoch-based segmentation to capture non-linear fashion shifts: theta_i(t) = E f_i ⊙ w(t) + Delta_E(t) f_i, and epoch-specific parameters Theta_ep.
• Incorporate non-visual temporal dynamics with beta_i(t) and beta_{C_i}(t) to separate visual from non-visual effects.
• Optimize with Bayesian Personalized Ranking (BPR) and a coordinate ascent procedure that alternates parameter fitting with fashion epoch segmentation Lambda.

Experimental results

Research questions

• RQ1 How can visual appearance and its temporal evolution be incorporated into a one-class collaborative filtering framework?
• RQ2 Can epoch-based segmentation improve capturing non-linear fashion dynamics over time compared to fixed-time models?
• RQ3 What is the impact of adding lightweight non-visual temporal dynamics on predictive performance and interpretability?
• RQ4 How do visually-informed models perform in warm- and cold-start settings on real-world fashion datasets?

Key findings

• The proposed TVBPR family outperforms state-of-the-art personalized ranking methods on two large Amazon Women’s and Men’s Clothing datasets.
• Epoch-based segmentation captures non-linear fashion evolution better than fixed-parameter time models.
• Incorporating visual features from deep CNNs via an embedding matrix improves recommendation quality in sparse, long-tail item settings.
• Temporal dynamics at the population level (fashion trends) substantially aid personalized rankings, with per-user drift being less impactful in their datasets.
• The model supports visualization of learned fashion trends over an 11-year span, illustrating complex, non-linear visual evolution.