[Paper Review] Exploring Generative Models for Joint Attribute Value Extraction from Product Titles

K S Roy, Tapas R. Nayak+1 more|arXiv (Cornell University)|Aug 15, 2022

Advanced Text Analysis Techniques4 citations

TL;DR

This paper proposes a novel generative approach for joint attribute value extraction (AVE) from product titles, formulating the task as text generation using two paradigms: word sequence-based and positional sequence-based. The method achieves state-of-the-art results on two benchmark datasets using T5 and BART models without task-specific tagging or architecture design.

ABSTRACT

Attribute values of the products are an essential component in any e-commerce platform. Attribute Value Extraction (AVE) deals with extracting the attributes of a product and their values from its title or description. In this paper, we propose to tackle the AVE task using generative frameworks. We present two types of generative paradigms, namely, word sequence-based and positional sequence-based, by formulating the AVE task as a generation problem. We conduct experiments on two datasets where the generative approaches achieve the new state-of-the-art results. This shows that we can use the proposed framework for AVE tasks without additional tagging or task-specific model design.

Motivation & Objective

• To address the limitations of sequence labeling approaches in attribute value extraction (AVE), which require task-specific tagging and are hard to scale for large attribute sets.
• To explore whether generative models can jointly extract attributes and their values without relying on predefined tag sets or separate models per attribute.
• To evaluate the effectiveness of two novel generative formulations—word sequence-based and positional sequence-based—for end-to-end AVE.
• To demonstrate that generative models can outperform existing tagging-based methods in both joint extraction and individual attribute/value prediction tasks.
• To investigate the robustness of the approach on sentences with single versus multiple attribute-value pairs.

Proposed method

• Formulate the AVE task as a text generation problem by transforming input product titles into structured output sequences of attribute-value pairs.
• Propose a word sequence-based paradigm where each attribute-value pair is generated as 'value; attribute' and separated by '|', enabling joint generation of both components.
• Introduce a positional sequence-based paradigm where each pair is represented as '[start_pos] [end_pos] attribute', with the model generating the span and corresponding attribute.
• Utilize pre-trained sequence-to-sequence models—T5, BART, WDec, and PNDec—as the generative backbone to decode the target sequence from the input title.
• Train the models end-to-end using cross-entropy loss to minimize the difference between generated and gold-standard attribute-value sequences.
• Apply post-processing to filter invalid outputs (e.g., missing start/end positions or incomplete attribute-value pairs) and evaluate using standard F1 scores.

Experimental results

Research questions

• RQ1 Can generative models effectively perform joint attribute and value extraction without task-specific tagging or model design?
• RQ2 How do word sequence-based and positional sequence-based generative paradigms compare in performance and robustness across different datasets?
• RQ3 Does the generative approach outperform existing sequence labeling and question-answering-based methods in AVE tasks?
• RQ4 How does the model perform on sentences with single versus multiple attribute-value pairs?
• RQ5 To what extent do generative models generalize beyond the annotated attribute-value pairs in the training data?

Key findings

• The proposed generative frameworks achieve state-of-the-art results on both AV-DATA-V1 and AV-MAE datasets, with T5 achieving 78.36 F1 on AV-MAE in the word sequence-based paradigm.
• In the AV-DATA-V1 dataset, BART achieves the highest F1 score of 64.81 in joint attribute value extraction under the word sequence-based paradigm.
• The positional sequence-based paradigm with T5 achieves 70.16 F1 on AV-MAE, outperforming BERT-NER (85.40 F1 for multi-value, but lower for joint extraction).
• Generative models show consistent superiority over tagging-based models like BERT-NER and SUOT, especially in handling multiple attribute-value pairs.
• The model's performance is robust to reordering of attribute-value pairs during training in the word sequence-based paradigm, indicating strong generalization.
• Despite generating some valid but unannotated attribute-value pairs (not in gold standard), the model maintains high precision and recall, suggesting strong generalization capability.