[Paper Review] Large Scale Product Categorization using Structured and Unstructured Attributes

Abhinandan Krishnan, Abilash Amarthaluri|arXiv (Cornell University)|Mar 1, 2019

Text and Document Classification Technologies4 citations

TL;DR

This paper proposes a deep learning approach for large-scale product categorization by unifying structured and unstructured product attributes into a single text-like representation, enabling effective use of convolutional neural networks. The method improves classification accuracy by 2.7% on a 6,000-class taxonomy, with significant gains on niche product types like tank tops and office boxes.

ABSTRACT

Product categorization using text data for eCommerce is a very challenging extreme classification problem with several thousands of classes and several millions of products to classify. Even though multi-class text classification is a well studied problem both in academia and industry, most approaches either deal with treating product content as a single pile of text, or only consider a few product attributes for modelling purposes. Given the variety of products sold on popular eCommerce platforms, it is hard to consider all available product attributes as part of the modeling exercise, considering that products possess their own unique set of attributes based on category. In this paper, we compare hierarchical models to flat models and show that in specific cases, flat models perform better. We explore two Deep Learning based models that extract features from individual pieces of unstructured data from each product and then combine them to create a product signature. We also propose a novel idea of using structured attributes and their values together in an unstructured fashion along with convolutional filters such that the ordering of the attributes and the differing attributes by product categories no longer becomes a modelling challenge. This approach is also more robust to the presence of faulty product attribute names and values and can elegantly generalize to use both closed list and open list attributes.

Motivation & Objective

• Address the challenge of extreme multi-class product categorization in dynamic eCommerce environments with thousands of classes and millions of products.
• Overcome limitations of traditional models that treat product data as a single text block or ignore structured attributes.
• Develop a unified representation that treats structured attributes and their values as unstructured text, enabling robust generalization across diverse product categories.
• Improve model performance by effectively leveraging both the presence and values of structured attributes using convolutional filters.
• Demonstrate that flat deep learning models can outperform hierarchical models in specific cases, particularly when using unified attribute representations.

Proposed method

• Represent each product’s structured attributes (e.g., 'sleeve_style: tank') as a single string by concatenating attribute names and values with a separator token.
• Feed the unified text representation into a Multi-CNN or Multi-LSTM model to extract deep features from unstructured and structured attributes jointly.
• Use separate word embedding spaces for each unstructured attribute (e.g., title, description) to capture domain-specific semantics.
• Apply 1D convolutional filters over the concatenated structured attribute string to extract patterns related to attribute presence and value semantics.
• Employ microbatching with a Redis buffer to optimize GPU utilization and reduce inference latency in a production microservices deployment.
• Use stratified sampling and dictionary pruning as implicit regularizers to improve generalization, especially for low-support classes.

Experimental results

Research questions

• RQ1 Can a unified representation of structured and unstructured product attributes improve classification accuracy in extreme multi-class product categorization?
• RQ2 Does combining structured attribute names and values into a single text string with a separator enable better generalization across product categories with varying attribute sets?
• RQ3 How do flat deep learning models compare to hierarchical models in terms of performance on large-scale product categorization tasks?
• RQ4 To what extent do convolutional filters effectively capture both the presence of specific attributes and the semantic meaning of their values?
• RQ5 What impact does using separate word embeddings per attribute have on model performance compared to a shared embedding space?

Key findings

• The proposed method improved overall F1 score by 2.7% on the evaluation set compared to baseline models, a significant gain in a 6,000-class classification problem.
• The Multi-CNN model achieved comparable performance to the Multi-LSTM model but with better parallelization, making it faster at both training and inference.
• The top five product types benefiting most from structured attributes were tank tops (+0.365 F1 gain), chemistry experiment kits (+0.317), office boxes (+0.257), outdoor flags & banners (+0.249), and shape sorting toys (+0.241).
• The use of a separator token between attribute names and values significantly improved model performance by preventing incorrect cross-attribute attention.
• Word embeddings as small as 50 dimensions performed nearly as well as 200-dimensional embeddings, suggesting that smaller embeddings can be effective with proper regularization.
• Trimming word dictionaries per attribute acted as an implicit regularizer and improved performance in some cases, particularly for low-support classes.