[Paper Review] Knowledge Enhanced Neural Fashion Trend Forecasting

Yunshan Ma, Yujuan Ding+4 more|arXiv (Cornell University)|May 7, 2020

Generative Adversarial Networks and Image Synthesis43 references4 citations

TL;DR

This paper proposes KERN, a knowledge-enhanced recurrent neural network for fine-grained fashion trend forecasting using time-series data from Instagram. By integrating internal similarity relations and external taxonomy-based affiliations into a deep learning framework, KERN outperforms baselines in predicting complex fashion trends across user groups, demonstrating superior accuracy in both half-year and one-year forecasts on a newly curated large-scale dataset (FIT).

ABSTRACT

Fashion trend forecasting is a crucial task for both academia and industry. Although some efforts have been devoted to tackling this challenging task, they only studied limited fashion elements with highly seasonal or simple patterns, which could hardly reveal the real fashion trends. Towards insightful fashion trend forecasting, this work focuses on investigating fine-grained fashion element trends for specific user groups. We first contribute a large-scale fashion trend dataset (FIT) collected from Instagram with extracted time series fashion element records and user information. Further-more, to effectively model the time series data of fashion elements with rather complex patterns, we propose a Knowledge EnhancedRecurrent Network model (KERN) which takes advantage of the capability of deep recurrent neural networks in modeling time-series data. Moreover, it leverages internal and external knowledge in fashion domain that affects the time-series patterns of fashion element trends. Such incorporation of domain knowledge further enhances the deep learning model in capturing the patterns of specific fashion elements and predicting the future trends. Extensive experiments demonstrate that the proposed KERN model can effectively capture the complicated patterns of objective fashion elements, therefore making preferable fashion trend forecast.

Motivation & Objective

• Address the challenge of predicting fine-grained fashion trends for specific user groups using data-driven methods.
• Overcome limitations of existing datasets that lack fine-grained fashion elements, long time spans, and user-level information.
• Develop a deep learning model capable of capturing complex temporal patterns in fashion trends beyond simple seasonal or linear trends.
• Incorporate both internal (similarity) and external (taxonomy-based affiliation) domain knowledge to improve forecasting performance.
• Generate actionable fashion trend reports by forecasting diverse fashion attributes across user demographics.

Proposed method

• Construct a large-scale fashion trend dataset (FIT) from Instagram, capturing time-series fashion element records and user demographics (age, gender, location).
• Design a Knowledge Enhanced Recurrent Network (KERN) that combines bidirectional GRUs for sequence modeling with attention mechanisms.
• Integrate internal knowledge via a triplet regularization loss to preserve semantic similarity between related fashion elements.
• Incorporate external knowledge using a message-passing mechanism on a hierarchical fashion taxonomy tree to update element embeddings.
• Train the model end-to-end using a multi-task loss combining reconstruction, triplet, and knowledge-aware regularization terms.
• Leverage user-specific embeddings to personalize trend forecasts, enabling group-wise predictions for different demographics.

Experimental results

Research questions

• RQ1 Can a data-driven model effectively forecast complex, fine-grained fashion trends across diverse user groups using social media data?
• RQ2 How does incorporating internal knowledge (e.g., similarity between fashion elements) improve trend forecasting performance?
• RQ3 To what extent does external knowledge (e.g., hierarchical taxonomy of fashion categories) enhance the model’s ability to generalize and predict trends?
• RQ4 How does the inclusion of user-level information affect the accuracy and personalization of fashion trend forecasts?
• RQ5 Can the proposed model generate comprehensive, interpretable fashion trend reports that reflect multi-dimensional fashion evolution?

Key findings

• The KERN model achieves superior performance in both half-year and one-year fashion trend forecasting compared to baseline models on the FIT dataset.
• Removing either internal or external knowledge significantly degrades model performance, with the most substantial drop observed in long-horizon predictions.
• The ablated KERN-IE model (without any knowledge) performs the worst, confirming that knowledge integration is crucial for handling complex fashion patterns.
• User-specific trend predictions are more accurate than global predictions, demonstrating the value of incorporating demographic information.
• Visualization results show that KERN effectively captures non-linear and complex trend patterns across fashion elements like color, pattern, and style.
• A comprehensive fashion trend report generated from KERN forecasts (e.g., for women in Paris) successfully captures multi-dimensional trends from category to detailed attributes like denim wash colors.