[Paper Review] Product Title Refinement via Multi-Modal Generative Adversarial Learning

Jianguo Zhang, Pengcheng Zou+6 more|arXiv (Cornell University)|Nov 11, 2018

Topic Modeling19 references4 citations

TL;DR

This paper proposes MM-GAN, a multi-modal generative adversarial network that generates concise, human-like product titles by integrating textual long titles, product images, and attribute tags. By framing title generation as a reinforcement learning task with a discriminator evaluating outputs for human-likeness, MM-GAN outperforms state-of-the-art baselines on ROUGE metrics, achieving 69.53 ROUGE-1, 52.38 ROUGE-2, and 65.80 ROUGE-L on a real-world E-commerce dataset.

ABSTRACT

Nowadays, an increasing number of customers are in favor of using E-commerce Apps to browse and purchase products. Since merchants are usually inclined to employ redundant and over-informative product titles to attract customers' attention, it is of great importance to concisely display short product titles on limited screen of cell phones. Previous researchers mainly consider textual information of long product titles and lack of human-like view during training and evaluation procedure. In this paper, we propose a Multi-Modal Generative Adversarial Network (MM-GAN) for short product title generation, which innovatively incorporates image information, attribute tags from the product and the textual information from original long titles. MM-GAN treats short titles generation as a reinforcement learning process, where the generated titles are evaluated by the discriminator in a human-like view.

Motivation & Objective

• To address the challenge of generating concise, accurate short product titles for mobile E-commerce displays, where long titles are often redundant or misleading.
• To overcome the limitations of text-only models that suffer from exposure bias and lack holistic evaluation by incorporating visual and attribute modalities.
• To improve title generation quality by modeling human-like judgment through a discriminative reward in a reinforcement learning framework.
• To enable the model to filter out irrelevant or conflicting terms (e.g., 'wild' and 'delicate' simultaneously) by leveraging multi-modal context.
• To establish a new benchmark for short product title generation by combining image, attribute tags, and long titles in a generative adversarial setting.

Proposed method

• MM-GAN employs a multi-modal encoder that processes long product titles via an LSTM, attribute tags via fully connected layers, and product images via a fine-tuned VGG16 to extract visual features.
• The generator uses a seq2seq architecture with attention to produce short titles based on fused multi-modal embeddings from text, image, and attributes.
• A discriminator is trained to distinguish between human-written and machine-generated short titles, providing a reward signal that encourages human-like fluency and accuracy.
• The training process follows a reinforcement learning paradigm where the generator is optimized based on the discriminator’s feedback, reducing exposure bias inherent in maximum likelihood estimation.
• The model is jointly trained using adversarial loss and a reinforcement learning objective, with the discriminator providing a holistic evaluation of generated sequences.
• Fine-tuning of VGG16 on product classification tasks enhances visual feature representation for better alignment with product semantics.

Experimental results

Research questions

• RQ1 Can incorporating visual and attribute information improve the quality of generated short product titles beyond relying solely on long textual titles?
• RQ2 Does a discriminative reward mechanism that evaluates full sequences in a human-like manner reduce exposure bias and improve fluency and relevance?
• RQ3 How does multi-modal integration affect the model’s ability to filter out conflicting or redundant terms present in long product titles?
• RQ4 To what extent does MM-GAN outperform text-only and single-modality baselines in generating concise, informative, and accurate short titles?
• RQ5 Can a generative adversarial framework effectively model the holistic, contextual understanding required for high-quality product title generation in E-commerce?

Key findings

• MM-GAN achieves the highest performance on all three ROUGE metrics—69.53 ROUGE-1, 52.38 ROUGE-2, and 65.80 ROUGE-L—on the test set, outperforming all baselines.
• The inclusion of image and attribute tags significantly improves generation quality, as shown by the performance gap between GAN (no modality fusion) and MM-GAN.
• MM-GAN successfully filters out irrelevant or contradictory terms (e.g., 'wild' and 'delicate' simultaneously) by leveraging multi-modal context, unlike FE-Net and Agree-MTL which retain such terms.
• The model generates more fluent and informative titles, as demonstrated in case studies where it preserves core product attributes like 'Artka', 'retro', and 'shirt' while omitting redundant or incorrect terms.
• The discriminator’s human-like evaluation capability enables the generator to produce more natural-sounding titles, reducing repetition and truncation common in MLE-based models.
• MM-GAN demonstrates superior generalization by effectively fusing multi-modal inputs, proving that visual and attribute signals are essential for accurate short title generation in real-world E-commerce.